- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes BQ-LoRA, a framework for adapting LLM agents with a single fixed-rank LoRA adapter. It combines behavior quotient balancing (BQB), which compares projected trajectory updates through their first-order effects on decision representations at shared reference histories and downweights redundant responses, with decision preserving compression (DPC), which compresses the resulting update under the rank constraint while controlling both parameter approximation error and first-order representation distortion. The paper formalizes a local behavioral equivalence relation and provides bounds on representation error, with a connection to conditional token distributions at fixed reference prefixes. Evaluation uses Qwen3.5-4B and Qwen3.5-9B on AppWorld and BrowseComp-Plus, comparing against full fine-tuning, single- and multi-adapter methods, and trajectory curation. The reported results show improved task performance and fewer interactions, supported by component ablations, alternative weighting and compression comparisons, and sensitivity analyses. Resource measurements indicate inference costs close to standard LoRA, with increased training time and memory.
Critical strengths
-
A coherent approach to behavior-aware training under a fixed rank budget. BQB and DPC use the same decision-response geometry to guide trajectory weighting and low-rank compression. This connects the treatment of redundant supervision with preservation of the resulting update effects, while retaining a single deployable adapter. This integration is consequential because it addresses a practical agent-adaptation problem without requiring capability-specific routing at inference.
-
Consistent empirical improvements supported by relevant ablations. BQ-LoRA achieves the highest reported mean across all eight performance metrics for both backbone sizes. On AppWorld Test-C, TGC improves over the strongest corresponding baseline by 3.03 and 3.20 percentage points for the 4B and 9B models, respectively. Component ablations, together with comparisons against text/parameter-based weighting and SVD compression, provide substantive evidence that both proposed modules contribute to the gains. These results strengthen the case for practical value beyond standard LoRA optimization.
Critical weaknesses
-
The implemented bandwidth rule does not adequately substantiate the balancing rationale. Appendix E.2 sets the bandwidth squared to the median strictly positive squared response distance. Consider a batch containing two groups of identical responses, with 60 and 4 trajectories respectively, separated by distance . This rule gives , so cross-group affinity remains , regardless of separation. Even with , the groups receive approximately 90.66% and 9.34% of the total weight, rather than approaching equal contributions. This does not contradict the conditional theoretical bounds, but exposes a gap between the bandwidth choice and the intended balancing behavior. Without controlled duplication experiments or diagnostics of actual affinities and weights, the paper leaves its central mechanism insufficiently validated.
-
The incremental contribution over established gradient-based data selection remains insufficiently demonstrated. Prior work already uses gradient information to identify useful and redundant supervision. In particular, GTP explicitly performs joint selection to reduce redundancy and evaluates on ALFWorld agent data. Although Table 6 includes a parameter-distance weighting baseline, it does not establish an advantage over such established joint-selection methods. The decision-response metric and its integration with compression are identifiable distinctions, but stronger comparisons are needed to determine their additional value. This omission materially limits my confidence in the significance of the proposed contribution.
Questions for authors
-
Clarification of the bandwidth rule. Is the median-positive-distance rule in Appendix E.2 the exact implementation, or are additional scaling or numerical safeguards used? For two internally identical response groups of unequal sizes, this rule keeps cross-group affinity at regardless of separation. How do the authors reconcile this behavior with the balancing rationale, and do observed affinities and group weights demonstrate meaningful redundancy correction during training?
-
Clarification of the weighting ablation. In Table 6, does “Parameter Weighting” use the same projected trajectory directions, normalization, density estimator, balancing exponent, and optimization procedure as BQB, changing only the distance metric? These details are necessary to determine whether the comparison isolates the contribution of the decision-response geometry.
-
Evidence needed to assess the incremental contribution. How does BQB improve upon gradient-based joint-selection methods such as GTP, which already addresses redundancy and evaluates on agent data? Can the authors provide a comparison under comparable data and computational budgets, or explain any concrete obstacle to such a comparison?
6 comments
To W2
GTP already addresses redundancy by jointly selecting training examples from gradient information and evaluates that procedure on ALFWorld. The additional contribution of BQ-LoRA is to measure overlap by the decision changes induced by an update, then use the same metric to preserve those changes under rank compression. GTP makes a subset decision for the student using teacher-gradient snapshots. BQB revisits the weights at each update as the policy changes, and DPC retains the balanced update in a single adapter.
A concrete AppWorld comparison illustrates the distinction. From 576 candidates, joint GTP selection retained a vacation-photo sorting trajectory and omitted a Spotify library cleanup trajectory. At the first BQ update, both trajectories contributed, with weights of 0.795 and 0.853 on a scale with mean one. Their projected gradients had cosine similarity 0.253, while their effects on the same reference decisions had cosine similarity 0.971. The example shows the information available to BQB when it weights the two records. The aggregate comparison below tests the value of the complete method.
For each backbone, both methods start from the paper training pools of 642 AppWorld trajectories and 668 BrowseComp-Plus trajectories. GTP jointly selects half the candidates for a rank-32 LoRA student. The student and the full-pool methods receive the same number of optimizer updates. The AppWorld columns report task completion and the retrieval columns report answer accuracy, all in percent.
| Backbone | Method | Test-N | Test-C | BM25 | Embedding |
|---|---|---|---|---|---|
| 4B | LoRA | 52.98 | 35.49 | 20.89 | 40.22 |
| 4B | GTP | 59.13 | 42.37 | 28.00 | 45.11 |
| 4B | BQ-LoRA | 60.91 | 44.04 | 29.11 | 46.44 |
| 9B | LoRA | 60.71 | 45.08 | 26.22 | 46.22 |
| 9B | GTP | 68.85 | 51.72 | 32.00 | 50.67 |
| 9B | BQ-LoRA | 70.44 | 53.32 | 33.11 | 51.78 |
GTP improves on full-data LoRA in all four columns for each backbone. BQ-LoRA adds 1.67 Test-C points on 4B and 1.60 on 9B over that gradient-based baseline. These differences compare the complete methods at matched student updates. Table 6 provides the complementary test of the weighting metric alone.
To Q1
Appendix E.2 gives the training rule exactly. In each batch we take the interpolated median of strictly positive squared response distances over unordered pairs to obtain . We use a bandwidth multiplier of one. The diagonal and zero distances are excluded from the median, while self-affinity and the affinity of identical responses both contribute one to the density. We raise density to the power and normalize the weights to mean one. There is no bandwidth floor or weight clipping. If all distances vanish, all trajectories receive weight one.
In the two-group example, increasing increases in the same proportion. Cross-group affinity therefore stays at . This explains why the 4B default moves the minority share from 6.25% to 8.46%. For training data, the tails of the affinity distribution are more informative than its median, which is largely set by the bandwidth rule. The table shows the off-diagonal 10th and 90th percentiles. Effective size is the number of equally weighted trajectories that would give the same weight concentration.
| Update | Affinity P10 | Affinity P90 | Max weight | Effective size / 64 |
|---|---|---|---|---|
| 4 | 0.095 | 0.914 | 9.14 | 29.77 |
| 5 | 0.305 | 0.795 | 1.85 | 61.27 |
| 6 | 0.171 | 0.826 | 9.96 | 17.68 |
| 7 | 0.272 | 0.824 | 11.97 | 21.54 |
| 8 | 0.109 | 0.840 | 7.03 | 35.19 |
| 9 | 0.038 | 0.909 | 11.05 | 23.34 |
The default gives nearly uniform weights on some updates and substantial reweighting on others. Effective size was 61.27 of 64 at step 5 and 17.68 at step 6. To check the direction of the correction, we also duplicated a single Spotify update while keeping the references fixed. Eight copies contributed 11.27% of the expanded batch by count. Their aggregate weight was 8.20% with the original weights frozen and 7.70% with densities recomputed. The training statistics establish when weights concentrate, and the controlled copies identify a reduction specifically attributable to redundancy.
To W1
The default bandwidth discounts redundant responses. Equal total weight across groups requires the additional small-overlap condition in Section 4.3 and α=1. In the 60-to-4 example every positive distance equals D. The rule therefore gives σT2=D2 and a cross-group affinity of e−1/2 for every D>0. The four trajectories account for 6.25% of the batch. Their weight increases to 8.46% at the 4B default α=0.75 and 9.34% at α=1, exactly as the calculation in the question predicts for the latter setting.
The same bandwidth rule behaves differently for tightly clustered responses that are distinct. For example, fix the distance between groups at 4. If each within-group distance is 1, then 1,776 of the 2,016 pairs are within-group. The median positive squared distance is now 1 and cross-group affinity falls to e−8. The table gives the share of total weight assigned to the four trajectories.
The relevant separation is the distance relative to the batch bandwidth. Exact duplicates contribute zero distances, which are excluded from the median that sets the bandwidth. The response geometry determines the amount of correction in each batch.
For a direct duplication test, we copied the update for a Spotify task about following classical artists, holding its 64 reference histories and the other 63 directions fixed. Freezing each original weight before renormalization controls for the fact that this trajectory already had a below-average weight. With eight copies, the group receives 8.20% with frozen weights and 7.70% after densities are recomputed. The additional 0.50-point reduction isolates the response to duplication itself. The total reduction relative to uniform weighting is 3.57 points.