acceptodds
Under review as a conference paper at ICLR 2027

GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuning

Abstract

Prompt-based continual learning (CL) offers a parameter-efficient way to adapt large language models (LLMs) across task sequences. However, existing methods often rely on task-aware inference and maintain an expanding set of task-specific prompts, leading to (1) severe performance degradation on earlier tasks when task identifiers are unavailable for prompt selection at inference time, and (2) limited scalability as task sequence grows. We propose GRID, a unified framework designed to address these challenges. GRID incorporates an output-space–aware decoding mechanism that enhances backward transfer by leveraging representative inputs and automatic label semantic normalization, alongside a gradient-guided prompt selection strategy that compresses less informative prompts into a single aggregated representation for scalable, memory-efficient continual learning. Extensive experiments on long-sequence and negative-transfer benchmarks show that GRID improves backward transfer, achieves competitive forward transfer, and substantially reduces prompt memory across encoder-decoder and decoder-only architectures, including T5, Qwen, and LLaMA.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, transfer learning, and lifelong learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_oNbGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies continual prompt tuning when task IDs are unavailable for prompt selection. It combines representative training examples, label normalization and constrained decoding with gradient-based prompt selection and merging. Experiments cover classification sequences, SuperNI, and a math/tool benchmark. The method reduces prompt storage and improves several reported accuracy and retention measures, with some accuracy trade-offs.

Critical strengths
  1. The paper addresses a practical limitation of accumulating task prompts: inference must remain usable without task-specific prompt routing while prompt length stays manageable.
  2. Evaluation includes multiple task orders and architectures, plus open-ended generation. These experiments broaden the evidence beyond fixed-label classification.
  3. The appendices provide runtime measurements, threshold sensitivity, and Random/FIFO comparisons. The cumulative ablation helps distinguish the retention gains associated with decoding from the storage savings of compression.
Critical weaknesses
  1. The test-time output-space information is unclear. Section 4.1 takes (x,Yi)(x,\mathcal Y_i) as input and constrains predictions to the corresponding Li\mathcal L_i. How is this candidate set obtained in a mixed task stream? A request-provided label set is a reasonable setting, but dataset-provided output spaces would supply information relevant to task identification. Baselines should receive the same label normalization and decoding constraints. Otherwise, the retention gains cannot be separated from output-space information and formatting corrections.

  2. The retention metrics need reproducible definitions. Section 6.3 defines FTC as a task count, yet Tables 4-5 report 77-95 for sequences containing 15 tasks. If these are accumulated task-time events, the aggregation must be stated. The relationship between average accuracy and BWT is also unclear: Table 4 reports 75.70 accuracy and -0.7275 BWT for ProgPrompt on T5-large/L1. These cannot both represent final average accuracy and standard BWT on a [0,1] scale. Please provide the metric formulas, units, and full evaluation matrices.

  3. Two algorithm definitions do not produce the stated behavior. The Section 4.1 formula softmax⁡(M⊙zt)\operatorname{softmax}(M\odot z_t) leaves positive probability on forbidden tokens. Hard masking requires excluding their logits, and multi-token labels need sequence constraints. Also, Eq. (3) excludes scores equal to the threshold. With a single retained prompt and zero score variance, both sets are empty, so Algorithm 1 keeps only the new prompt. Please specify the actual masking, tie handling, and initialization or compression-trigger rules. These may be presentation errors, but the described algorithm is not sufficient to reproduce the system.

  4. The gradient score has not been validated as a measure of historical redundancy. A small current-task gradient can indicate that a prompt is irrelevant to the new task while still essential to an old one. The reported prompt similarities and Table 25 are useful, but do not establish safe knowledge preservation. Tables 6 and 26 also give slightly better BWT without gradient selection, contrary to the statement that full GRID is best. A matched-budget comparison and historical-task losses immediately before and after merging would better test the proposed mechanism.

  5. Several conclusions overstate the reported results. Section 6.1 claims approximately +3.8/+3.6 points on Flan-T5-base/large under negative transfer, whereas Table 21 gives +1.15/-0.74 points. GRID has lower average accuracy on four of that table's six backbones, despite its caption claiming consistent improvement. These discrepancies need correction. The paper averages three runs, but provides no dispersion for the small selection-strategy differences, which further limits the mechanism claims.

Questions for authors

See weaknesses

Reviewer_jUtBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GRID, a framework for continual soft-prompt tuning with a frozen language model. It addresses two practical challenges: inference without task-ID-based prompt selection and the growing cost of maintaining task-specific prompts. GRID combines representative training-sample selection, semantic label normalization, output-space-constrained decoding, and gradient-guided prompt compression. Existing prompts are scored using gradient norms on the current task; higher-scoring prompts are retained, while lower-scoring prompts are merged through weighted averaging.

Experiments cover multiple task orders, negative-transfer sequences, open-ended SuperNI tasks, and a mathematical reasoning and tool-selection benchmark, using T5, Qwen, and LLaMA backbones. GRID reports improved backward transfer and competitive average task performance while retaining approximately five prompts instead of fifteen. On T5-large, reported average accuracy increases from 77.8 for SHLPT to 79.2. The ablations indicate that constrained decoding accounts for most of the retention improvement on classification tasks, while gradient-based compression primarily reduces prompt storage.

Critical strengths
  1. The paper addresses a relevant problem in managing accumulated task adaptations: Continual adaptation creates practical questions about how to combine, retain, and compress task-specific modules. Evaluating a shared prompt pool without task-ID-based prompt routing is useful, particularly when deployment requires a unified inference interface.
  2. The experimental coverage is broad: The paper evaluates encoder-decoder and decoder-only architectures, multiple task orders, deliberately dissimilar task sequences, and extensions beyond standard classification. The SuperNI and Math–Tool experiments help assess whether the approach remains useful when finite-label decoding is insufficient.
  3. The ablations provide useful insight into the components’ distinct roles: The results suggest that output constraints substantially improve classification retention, while prompt compression preserves approximately the same BWT with fewer stored prompts. Comparisons with FIFO and random selection, together with prompt-length, queue-size, and threshold studies, provide additional evidence about the method’s trade-offs.
  4. The compression mechanism is relatively simple and requires no backbone updates: Retaining selected prompts and merging others through weighted averaging is straightforward to incorporate into an existing prompt-tuning pipeline. The accompanying implementation also makes the approach more inspectable.
Critical weaknesses
  1. The scope of task-agnostic inference needs sharper qualification: The paper explicitly defines task-agnostic inference as the absence of task IDs for prompt selection. However, label identification and constrained decoding still use a candidate label set associated with the input. Such information can substantially identify the underlying task. The source of these labels, the reliability of the identification module, and its behavior on genuinely mixed inputs are insufficiently evaluated. The current evidence therefore supports a narrower setting than fully task-unknown inference.
  2. The principal retention gain is not clearly separated from output-format correction: In the ablation, removing gradient selection changes T5-large BWT from −0.3511 to −0.3490 and Qwen BWT from −0.4418 to −0.4393, while removing constrained decoding causes a much larger deterioration. This supports compression with little loss of performance, but does not establish that gradient selection itself improves knowledge retention. Because the backbone and previous prompts remain frozen, measured degradation can include prompt interference and invalid label generation. Strong instruction-based or label-scoring baselines, using the same output information and constraints, are needed to establish the additional continual-learning benefit.
  3. The gradient-based compression rationale remains insufficiently validated: A small gradient on the current task does not necessarily indicate redundancy: a prompt may be irrelevant to the current task while remaining important for an earlier one. Similarly, a large gradient can reflect conflict rather than useful distinct knowledge. The reported similarity observations provide motivation, but do not establish that weighted averaging preserves previous functionality. FIFO and random selection also outperform GRID on some accuracy or BWT comparisons. Furthermore, the claimed fixed-capacity setting should be connected more explicitly to the selection rule and its behavior beyond fifteen tasks.
  4. Several metric definitions and result descriptions require reconciliation: FTC values exceed the fifteen tasks in a sequence, indicating that they cannot represent the number of distinct forgotten tasks at the final checkpoint. The aggregation across learning stages needs an explicit definition. The evaluation checkpoints and averaging conventions for accuracy and transfer metrics also need clarification. Some textual claims conflict with the tables: the complete model does not achieve the best BWT in the component ablation, and negative-transfer accuracy does not improve across all backbones as stated in parts of the paper.
  5. The practical advantage for modern LLM adaptation is not yet convincingly established: Reducing prompt storage from 600 KB to 200 KB is a small absolute saving relative to model memory. Shortening the inference prefix could be more valuable, but the reported speedups need controlled end-to-end measurements, including label identification and decoding costs. Training is slower because of gradient scoring. The newer benchmarks provide useful evidence, but results remain mixed—for example, GRID improves retention while reducing average accuracy on Qwen in MTCL-Bench. Stronger modern adaptation baselines and longer task streams would help establish the significance of the proposed trade-off.
Questions for authors
  1. What information is available at inference time? How is the candidate label set obtained for each input? Is it inferred from the input alone, supplied externally, or retrieved using dataset metadata? Please distinguish information used for prompt selection from information used for label selection, and report label-identification accuracy and overhead.
  2. How much improvement remains when output processing is matched across methods? Could you compare GRID with strong instruction-based and prompt-accumulation baselines using identical label normalization, constrained decoding, and input information? This would help separate output-format correction from improved continual adaptation.
  3. What ensures that compression preserves older task functionality? Can you evaluate whether low current-task gradient scores predict the effect of removing or merging a prompt on earlier tasks? Please also clarify how the stated fixed capacity is enforced and whether the prompt pool remains controlled over substantially longer sequences.
  4. Could you specify the evaluation metrics completely? Please provide the formulas, checkpoints, and averaging procedures for accuracy, BWT, FWT, and FTC, including why FTC can exceed the number of tasks. Raw task-by-training-stage performance matrices and uncertainty estimates would help reconcile the reported results.
  5. Could you provide a controlled end-to-end efficiency comparison? Please include preprocessing, gradient scoring, label identification, and inference costs, and document the prompt-memory accounting for each backbone. When comparing different prompt lengths, is the amount of retained input text held constant, particularly under the 256-token sequence limit?
Reviewer_tHFqOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies continual prompt tuning under task-agnostic inference and proposes GRID. The method first trains a task prompt on class-balanced representative samples. It then scores old prompts with gradient norms and merges low-scoring prompts into one weighted aggregate prompt to bound the prompt pool. At inference time it identifies labels with rules or Phi-3.5 and applies constrained decoding. Experiments on T5, Qwen and LLaMA models and several 15-task sequences report accuracy, BWT and forgotten task count.

Critical strengths

The paper addresses two practical problems at the same time, task-agnostic inference and prompt pool growth.

The ablation honestly shows which components matter.

Critical weaknesses

The task-agnostic claim is weakened by the use of per-task label sets and an external label classifier.

The key baseline lacks the same label constraint, so the comparison is biased.

The compression component does not improve retention, and the merged prompt is not analyzed.

The formatting of Reference and AI Usage Statement is bad.

The table in Intro is meaningless and easy.

There is a big problem for the formatting of all pictures.

Questions for authors

How is the candidate label set obtained at test time? If it comes from task metadata, how is this different from a task ID?

Can the paper add a ProgPrompt baseline with the same constrained decoding?

Are the representative samples kept after training? Are they used at inference?

Why does removing gradient selection give slightly better BWT? What is the actual benefit of the selection component?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.