QPU-PEFT: Feedback-Driven Configuration Selection with Reliability-Gated Ising Retrieval

Parameter-efficient fine-tuning enables large language models to adapt to downstream tasks with a small number of task-specific parameters. Quantum-inspired unitary parameterizations offer compact representations of update directions, enabling methods such as Quantum-PEFT (Q-PEFT) to further reduce adapter parameters. However, Q-PEFT performance depends on interacting structural and training choices, yet evaluating each configuration requires costly GPU fine-tuning. This makes effective use of limited validation feedback essential for joint configuration optimization. We propose QPU-PEFT, a feedback-driven framework for budgeted Q-PEFT configuration selection. At each iteration, QPU-PEFT refits a quadratic surrogate from validation feedback, formulating its acquisition objective as a QUBO and an equivalent Ising Hamiltonian, which are used to retrieve the next configuration for GPU evaluation. We also develop a support-and-stability gate to accept QPU proposals after evaluation and otherwise invoke deterministic classical retrieval. We evaluate proposals on real QPUs. On TianYan-176, the gate accepts a hardware proposal, which differs from the deterministic classical route and satisfies a protocol-level 0.5-point BLEU/ROUGE-L non-inferiority criterion over three paired Qwen2.5-1.5B/E2E retraining seeds not used during selection; TianYan-287 separately invokes deterministic fallback. In single-seed offline replay on Qwen2.5-0.5B, Refitted Ridge-LCB reduces five-seed-mean regret by 39.8% relative to structured GP-EI and remains significantly better than RF-EI, TPE, and random search. On GPT-2/E2E-NLG, five paired full-budget retrainings show gains of 2.638 BLEU and 1.339 ROUGE-L over a learning-rate- and parameter-matched control. Finally, our separate resource-constrained selection reduces validation loss with 33.4% fewer trainable parameters and 52.4% less training time.