- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
For LLM-based sequential recommendation, an LLM planner routes users via 17 hand-built features to existing augmentation views (L2Aug, GenPAS prefix, AsarRec dual) under a fixed record budget. Candidates are scored by one-epoch Qwen3-4B fine-tuning. The validation-best of nine (three revision rounds) beats the best single method on Amazon Beauty, Toys, Sports (R@10 +4.59%, +12.35%, +18.30%), fixed-total routings, and a no-feedback planner.
Critical strengths
Matched-exposure controls (Table 3) fix method totals while shuffling who gets which view, isolating user routing from method mix. Toys and Sports R@10 gains are sizable (0.0828 vs. 0.0771, 0.0569 vs. 0.0480).
The policy is an inspectable rule list (Beauty: cluster_id==1 routes to GenPAS prefix, else AsarRec dual, p. 19), and the planner sees only aggregate validation metrics (p. 14).
Critical weaknesses
The LLM agent is not shown to beat picking the validation-best of nine candidates (pp. 6, 14), since baselines are single runs without budget-matched non-LLM search. Without feedback, the planner trails uniform AsarRec (Beauty 0.0770 vs. 0.0784, Sports 0.0456 vs. 0.0481, Tables 2–4), and displayed revisions do not improve test scores even when Sports validation rises (Tables 5–6), so gains may come from nine-try selection.
There are no SDs or significance tests, and Tables 3–5 omit seed counts. The same Beauty policy scores R@10 0.0800 (Tables 6–7) and 0.0820 (Tables 2, 5), so NDCG claims on p. 9 and p. 15 conflict. Beauty controls (0.0685–0.0717) also trail both uniform methods (0.0730, 0.0784), suggesting variance above the +0.0036 margin or a protocol difference.
Minor:
Every user gets one appended record (p. 6), so the policy reduces to method assignment. Missing: no-augmentation row, no-transfer control, and compute cost (pp. 6, 8). Related work omits close prior work: AdaAug (2022), LLM-AutoDA (2024), and DataEnvGym (2025).
Questions for authors
How does non-LLM search with the same nine-candidate budget (e.g., enumerating cluster-to-method assignments, random routings) compare?
What are per-seed standard deviations? Why is the Beauty Ref 0.0800 in Tables 6–7 but 0.0820 in Tables 2 and 5?