- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Query-Adaptive Skill Gating (QASG), a query-conditioned gating framework that can control the magnitude of the injected parametric skill adapters. This paper introduces two metrics, uncertainty and sensitivity, to determine whether the base model needs the skill adapter and how much the LoRA can help. They fine-tune Doc-to-LoRA's hypernet on skill-solution pairs, with solutions mostly generated by DeepSeek-V4-Flash. QASG demonstrates performance improvement over uniform, fixed scaler and latenskill and textskill baselines.
Critical strengths
-
This paper investigates an interesting and timely topic: when and at what strength can we apply the parametric skill adapters.
-
The experimental results are convincing, with supportive ablation and analysis.
Critical weaknesses
-
The paper says, "we release the first large-scale public training corpus for skill-conditioned hypernetworks", but this doesn't appear in the supplementary materials, and there is no anonymous link.
-
QASG introduces extra inference latency on uncertainty and sensitivity probes.
-
Latentskill has an official checkpoint, this paper doesn't fully clarify why and from which checkpoint (or just randomly initialized) they fine-tuned Latentskill.
-
QASG only tests on a Doc-to-LoRA-based hypernet, without testing its generalization ability over other hypernets like SHINE or Latentskill.
-
The combination formulation of two signals needs more elaboration or ablation.
Questions for authors
- Can QASG be directly added to LatentSkill or SHINE without training?
5 comments
W3. LatentSkill checkpoint initialization and continued training
Thank you for pointing out the availability of the official checkpoint and asking which weights we used and why we continued training.
- Qwen3-8B. We initialize the skill compiler from LatentSkill’s official pretrained weights. The archive is
latentskill_pretrain_qwen3_8b/pretrain.tar.gz, and the extracted directory ischeckpoints/latentskill_pretrain_qwen3_8b/pretrain/. Official LatentSkill checkpoint - Qwen3-4B. Since LatentSkill did not publicly release a pretrained skill compiler checkpoint for Qwen3-4B, we trained the Qwen3-4B skill compiler using LatentSkill’s publicly released skill pretraining data. LatentSkill skill pretraining data
For task adaptation, we continue training the LatentSkill compiler on the same corpus of skill–solution trajectories used in the QASG experiments. This adapts the compiler to the skills and queries in our evaluation and gives both methods the same task supervision. LatentSkill’s skill compiler is based on SHINE.
We will clarify the compiler’s architectural provenance and specify the initialization or pretraining resources used for each model size. Thank you for highlighting this detail, which matters for interpreting and reproducing the baseline comparison.
W4 / Q1. Compatibility with additional skill generators
Thank you for asking whether QASG can be applied beyond the Doc-to-LoRA hypernetwork used in our main experiments. We evaluate QASG with two additional skill-generation methods, LatentSkill and SHINE. Adding the QASG gate requires no additional gradient updates to either hypernetwork.
QASG controls the adapter at inference time, after skill generation and before answer generation. It can therefore be added to an existing skill-generation pipeline as long as the resulting adapters support injection-strength scaling.
For the SHINE experiment on Qwen3-8B, we initialize the hypernetwork from the official pretrained weights and continue training on the same skill–solution corpus used for QASG and LatentSkill. As with LatentSkill, this adapts the generator to the skills and queries in our evaluation and ensures that the methods receive the same task supervision. We then freeze the adapted SHINE generator and reuse the same adapters to compare its original inference procedure with QASG gating. SHINE-Pretrain
For each of LatentSkill and SHINE, we freeze the generator and hold the rest of the setup fixed. We vary only the strength-control strategy, comparing the generator’s native inference procedure with QASG’s dynamic scaling.
Table 2. QASG performance with different frozen skill generators (Qwen3-8B, accuracy in %)
| Skill generator | Inference strategy | CHAMP | TheoremQA | MedCalc-Bench | LogicBench | BigCodeBench | Avg. |
|---|---|---|---|---|---|---|---|
| LatentSkill | Native inference | 54.0 | 53.5 | 41.0 | 76.0 | 33.0 | 51.5 |
| LatentSkill | + QASG (ours) | 60.1 | 59.0 | 48.0 | 82.0 | 37.0 | 57.2 |
| SHINE | Native inference | 52.1 | 52.0 | 40.5 | 74.0 | 32.0 | 50.1 |
| SHINE | + QASG (ours) | 57.5 | 57.5 | 46.5 | 79.5 | 35.5 | 55.3 |
Adding QASG improves macro-average accuracy by 5.7 percentage points for LatentSkill and 5.2 percentage points for SHINE compared with their respective native inference procedures. These results show that QASG can improve average accuracy with either generator frozen. We will include the full experiments in the final manuscript. Thank you for suggesting this test of QASG across skill generators.
W5. Rationale and ablation evidence for combining the two signals
Thank you for raising this question. The original manuscript gave limited detail on the motivation and ablation evidence for the gate. We provide the rationale and results for the conditional coupling and tanh mapping below.
1. Rationale for conditionally coupling the two signals
Base-model uncertainty sets the baseline injection strength, and adapter sensitivity adjusts the gain relative to that baseline. The gate takes the form
Sensitivity’s marginal effect on the combined score is . The derivative captures how the sensitivity contribution depends on uncertainty.
- When the base model is highly confident (), the additional gain tends to zero at every sensitivity level. This limits the risk of excessive skill intervention on easy queries.
- Higher base-model uncertainty allows a wider range of sensitivity-based adjustments, giving the skill more room to correct errors.
An additive combination contributes independently of and therefore lacks this constraint. When , low sensitivity can yield a relatively high injection strength even on a query with low uncertainty.
In Figure 3(c), the negative association between sensitivity and the oracle scale is strongest in the high-uncertainty group (, compared with 0.19 and 0.17 in the medium- and low-uncertainty groups, respectively). In Figure 4(a), higher oracle scales also concentrate in the high-uncertainty, low-sensitivity region. Both patterns are consistent with the direction of the gate’s adjustments.
2. Rationale for the tanh mapping
The tanh mapping has two properties that motivate its use.
- Smooth, bounded output. Tanh smoothly maps nonnegative inputs into , which is consistent with the interpretation of the injection coefficient. Its slope varies smoothly. Clipping also yields a continuous output, with abrupt changes in slope at the clipping points.
- Saturation. The derivative decreases as the scale rises. This compresses increments at high injection strengths and limits the risk of excessive interference from the skill.
3. Ablation results
We compare three gates using the same adapters and signals to assess the contribution of conditional coupling and compare two mappings within the additive structure. Full QASG retains the parameters fixed in advance in the original paper. Following the same ablation protocol, we independently select for each additive baseline to maximize accuracy on the 100-question validation set used to calibrate the signals.
- Additive combination with tanh uses , the “w/o coupling” ablation in the original manuscript.
- Additive combination with clipping uses .
Table 3. Comparison of gates using the same signals (Qwen3-4B, accuracy in %)
| Gate | CHAMP | TheoremQA | MedCalc-Bench | LogicBench | BigCodeBench | Avg. |
|---|---|---|---|---|---|---|
| Additive combination + tanh | 55.2 | 54.0 | 44.8 | 76.5 | 30.0 | 52.1 |
| Additive combination + clipping | 54.2 | 53.1 | 45.7 | 75.2 | 30.6 | 51.8 |
| Full QASG (ours) | 60.1 | 58.8 | 49.4 | 78.3 | 31.4 | 55.6 |
- Conditional coupling. Full QASG exceeds the additive gate with clipping by 3.8 percentage points. With tanh held constant, it exceeds the additive gate by 3.5 percentage points, supporting the contribution of the conditional coupling.
- Mapping within the additive structure. The additive gate with clipping achieves 0.3 percentage points lower macro-average accuracy than the additive gate with tanh.
We will add the derivation and ablation results to the methods discussion in the final manuscript.
Thank you for reading our paper carefully and for providing detailed, concrete feedback. We appreciate your recognition of the importance of query-level skill strength control and of the evidence from our experiments and ablations.
W1. Release timing for the training corpus
Thank you for asking about data availability and reproducibility. As stated in the original manuscript’s Reproducibility Statement, we plan to release all code and training data upon acceptance.
The phrase “we release” in the main text could suggest that the corpus is already public. We will revise the wording throughout the manuscript to clarify when the complete corpus will be available.
We have added
QASG_training_trajectory_examples.zipto the supplementary materials for review. It contains 1,500 skill–solution trajectories, with 500 examples each from BigCodeBench, CHAMP, and TheoremQA. The appendix also describes the composition, construction, and verification of the corpus.We will publicly release the complete set of resources upon acceptance to support reproducibility. Thank you for flagging the ambiguity in the original wording.
W2. Inference overhead and the accuracy–latency trade-off
Thank you for raising inference overhead, an important consideration for deployment. Computing reflection entropy and adapter sensitivity adds inference latency. Our experiments show that the reflection budget can be adjusted to trade off accuracy against latency while retaining some of the accuracy gains at lower overhead.
Using the same experimental setup as in the main paper, we obtain the following results on Qwen3-4B with the default 200-token reflection budget.
QASG omits the retrieved skill document from the final answer prompt. This reduces the average input length for final answer generation from 1,556 to 112 tokens. Processing the shorter input offsets part of the probe overhead.
To quantify how much of the accuracy gain is retained at different latency budgets, we added an ablation over reflection lengths.
Table 1. Accuracy and end-to-end latency at different reflection budgets (Qwen3-4B)
We agree that reducing the overhead further is an important direction. We will investigate lighter-weight uncertainty estimators and faster approximations for the sensitivity computation.
Thank you for suggesting this ablation. We will include these results and the associated trade-offs in the computational cost analysis of the final manuscript.