- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates functional transfer from quantized edge large language models under a limited text-only query budget. It proposes CLIQ, which partitions a public instruction pool into semantic regions, filters out regions with insufficient sample support, and uses an independent generator to produce an equal number of queries for each retained region. Responses collected from the target model are then used to fine-tune a student model via LoRA. The evaluation covers off-device Qwen and Llama configurations, as well as two Gemma teachers deployed on a smartphone. Student outputs are evaluated primarily using BERT-F1, BLEU, and ROUGE against the original Alpaca reference responses. The reported results show that CLIQ generally outperforms random querying across the evaluated configurations and exceeds two diversity-aware selection baselines in the main comparison using a 4B student.
Critical strengths
- The complete pipeline demonstrates clear and practically meaningful empirical gains under a limited query budget.** With the same target model and 500 training steps, CLIQ achieves a BERT-F1 score of 82.44±0.07 for the 4B student using 495 target queries, outperforming OQ (77.92±0.09) and K-Center Greedy (80.45±0.08), both of which use 500 queries. Table 4 reports consistent improvements across two Llama target configurations. These differences substantially exceed the reported training standard deviations across three runs with fixed query sets, providing valuable empirical evidence for the importance of query construction under limited target access.
- Real-device validation provides concrete evidence in an edge deployment setting.** The paper collects target responses on a Pixel 10 Pro Fold using the Tensor G5 NPU and LiteRT-LM, covering two Gemma teachers and two student sizes. Under a common 200-step training protocol, CLIQ outperforms OQ on all six reported metrics across all four teacher–student combinations.
Critical weaknesses
- The paper does not sufficiently explain the additional value of extracting the edge target given the resources available to the attacker. It uses Qwen3-30B-A3B-Instruct-2507 to generate queries, then collects supervisory responses from a 7B Qwen model or smaller phone-deployed Gemma models. Parameter count alone does not establish relative capability, and using an independent generator is consistent with the stated threat model. However, the experiments primarily evaluate student outputs against Alpaca references, without demonstrating that the target provides domain knowledge or customized behavior that the attacker’s existing generator cannot readily supply. Nor do they compare against directly training the student on the generator’s responses under a comparable training budget. Thus, the results show that target responses can improve student performance, but do not establish the practical benefit of accessing the edge target over using the generator already available to the attacker.
- The paper does not sufficiently establish the incremental value of CLIQ’s region-coordination design over ordinary generative querying. CLIQ combines semantic clustering, equal query allocation across regions, representative-example retrieval, and instruction generation. However, OQ, K-Means Representative, and K-Center Greedy in Table 2 all use existing instructions from the public pool, whereas only CLIQ constructs new queries using a generator. Its 1.99-point BERT-F1 advantage over K-Center Greedy therefore reflects changes in both query organization and query generation, making it difficult to distinguish the contribution of regional coordination from the benefits of generating new tasks. Given that prior work cited in the paper, such as Self-Instruct, has already established the use of synthetic instructions for training, a generative baseline using the same generator, comparable generation resources, and matched target-query and student-training budgets, but without regional coordination, is crucial for assessing the method’s incremental value. I believe the current evidence supports the superiority of the complete CLIQ pipeline over the evaluated selection-based baselines, but does not yet establish additional benefits from its core coordination design. This issue warrants further discussion.
Questions for authors
- Given that the attacker already has access to Qwen3-30B-A3B-Instruct-2507, what additional capability or practical benefit motivates extracting the evaluated edge targets? Have you compared training the student on target responses with training it on the generator’s responses to the same queries under a matched student-training budget? If this comparison is unavailable, please clarify what target-specific value the current evaluation establishes.
- Have you evaluated a generative-query baseline using the same generator, comparable generation resources, and matched target-query and student-training budgets, but without semantic-region coordination? If such results are available, how much of CLIQ’s advantage remains? Otherwise, please clarify whether the current evidence establishes a benefit from coordination beyond that of generating new queries.