acceptodds
Under review as a conference paper at ICLR 2027

Chain2Parallel: Post-Hoc Parallelization of T5 Classifier Chains for Few-Shot Multi-Label Classification

Abstract

Multi-label text classification (MLTC) aims to assign a set of correlated labels to each document, a task that becomes particularly challenging when labeled data is scarce. Autoregressive classifier chains model these dependencies effectively, but their sequential label decoding incurs substantial inference cost. We propose Chain2Parallel, a post-hoc parallelization method that transforms a completed, task-specialized T5 Chain into an efficient, single-pass classifier. By leveraging the frozen label-conditioned encoder and extracting its label-aligned sentinel representations, Chain2Parallel predicts all labels in parallel via a lightweight shared classification head. Across three few-shot benchmarks, Chain2Parallel matches or improves upon the original chain's classification performance while substantially reducing inference latency, memory consumption, and model footprint.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multi-label text classification, few-shot learning, classifier chains, parallel inference, encoder representations, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dW12OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper converts a T5 classifier chain into a parallel classifier by freezing its encoder and training a shared MLP on label-aligned sentinel states. Across five splits, AvgF1 is maintained on EURLEX and AAPD and improves over the local chain on Reuters. Removing the decoder reduces deployed parameters by 55.7% and gives 50.9–214.5× faster batch-one inference under the reported implementation.

Critical strengths
  1. The conversion offers a useful deployment option for existing CHAIN checkpoints. It preserves aggregate performance on EURLEX and AAPD while substantially reducing inference cost. Both timing paths include fresh encoder passes, and the chain baseline uses decoder caching.
  2. The ablations clarify what matters, i.e., label-aligned sentinel states consistently outperform pooled features, while sharing and MLP gains vary by dataset. The representation analysis also controls for shared document and class-mean components rather than relying on raw correlations alone.
Critical weaknesses
  1. The benefit of task-specific chain training is not fully isolated. Chain2Parallel exceeds DirectParallel-GP by 2.9/1.5 AvgF1 points on EURLEX/AAPD but trails by 0.3 on Reuters, contradicting the introduction’s blanket claim. Frozen GP/DA/CS readouts would clarify the later stages’ contributions. The Reuters gain also needs qualification because the local source chain underperforms the published result.
  2. The efficiency comparison mainly supports converting an existing chain. For a new training pipeline, DirectParallel-GP offers the same deployment topology and competitive accuracy. Matched deployment and training-cost comparisons, together with an EncT5/T5Enc comparison, can better establish the method’s practical advantage.
Questions for authors
  1. Please report split-level paired AvgF1 differences and confidence intervals against both the local chain and DirectParallel-GP. Does the Reuters improvement persist with stronger source checkpoints? Please also revise the introduction’s claim about the direct baselines.
  2. Please compare identical shared heads trained on frozen GP, DA, and CS states, with matched inputs and model selection. Please include feature-extraction and head-training time.
  3. Can you report latency and footprint alongside DirectParallel-GP and clarify when chain training is worth its additional cost? A matched EncT5/T5Enc comparison would also help. Any optimized-chain comparison should preserve the sentinel prefixes required by subsequent decisions.
2 comments
AuthorsOpenReview ↗

Thank the reviewer very much for all insightful comments.

Response to Reviewer dW12

Q1: This comment asks for (i) paired results, (ii) a correction to the introductory comparison, and (iii) clarification of whether the Reuters gain extends to stronger source checkpoints. We address them in turn:

  1. Paired Results. The table below reports split‑level paired AvgF1 differences, their means, and two‑sided 90% Student‑t confidence intervals (Chain2Parallel minus reference; five matched splits).
DatasetReferenceSplit 1Split 2Split 3Split 4Split 5Mean90% CI
EURLEXT5 Chain+1.02−0.64+1.01−0.33+0.06+0.22[−0.51, +0.95]
EURLEXDirectParallel‑GP+1.29+1.30+5.58+1.07+4.94+2.84[+0.72, +4.96]
AAPDT5 Chain−0.23−0.59−0.84+0.66+0.98−0.01[−0.76, +0.75]
AAPDDirectParallel‑GP−0.91+2.63+1.31−1.10+5.85+1.56[−1.18, +4.29]
ReutersT5 Chain+11.01+7.69+5.57+9.03+7.57+8.17[+6.26, +10.09]
ReutersDirectParallel‑GP+2.76−1.15−3.19+0.53−0.26−0.26[−2.35, +1.82]

These results support our main finding: conversion preserves aggregate chain performance on EURLEX/AAPD. For Reuters, Chain2Parallel outperforms our reproduced T5 Chain across all five splits. These paired results have been added to the Appendix.

  1. Introductory Wording. We revise the introductory comparison to state its scope explicitly, replacing “They however are obviously inferior…” with “They however are inferior to T5 Chain and our method in the majority of cases”. Recall that in Table 1 there are 4 metrics, 2 DirectParallel baselines, and 3 datasets, yielding a total of 24 (4 × 2 × 3) cases; T5 Chain performs better on 20 out of 24.

  2. Scope of the Reuters Gain. The large Reuters improvement of Chain2Parallel is observed against our locally trained T5 Chain checkpoint, which failed to reproduce the original published performance. Whether this large gain persists when converting from the stronger official T5 Chain checkpoint remains unclear, as the learning rate and random seed settings are not available from their paper and released code. We have revised Section 4.2 (Main Results) to explicitly state that the Reuters gain is relative to its locally trained source T5 Chain. The accuracy retention and deployment savings on EURLEX/AAPD establish our method's practical value independently of the Reuters improvement.


Q2/W1: The concerns are (i) the incremental benefit of DA/CS for the frozen readout, and (ii) the cost of extracting features and fitting the head. We address these separately:

  1. The benefit of DA/CS. We trained the same shared MLP on frozen GP and CS representations across all three datasets, with matched inputs and head selection. This directly evaluates DA/CS's contribution to the features used by the parallel predictor (three matched splits per dataset; AvgF1, mean ± sample SD).
DatasetWithout DA/CSWith DA/CSGain
EURLEX50.83 ± 4.1968.72 ± 0.66+17.89
AAPD33.72 ± 3.1254.46 ± 0.14+20.74
Reuters67.54 ± 1.5279.11 ± 2.06+11.57

This table shows that removing DA/CS substantially lowers performance on every dataset, supporting the incremental value of task‑specific chain training. The comparison of adaptation routes in our paper’s Table 1 is a separate result: Chain2Parallel has higher mean AvgF1 on EURLEX/AAPD, while DirectParallel‑GP is slightly ahead on Reuters. Its strong Reuters result shows that parallel adaptation is also effective there; Reuters’ sparse co‑occurrence and long‑tailed labels may help explain why it remains competitive. We have added this table to the Appendix.

  1. Feature‑extraction and head‑fitting costs. Readout training adds little conversion cost. For CS conversion on the same RTX 4090D, train/validation sentinel extraction takes 10.76 ± 1.13 seconds and head fitting takes 2.28 ± 1.29 seconds (five main EURLEX splits).
AuthorsOpenReview ↗

Continue ...


Q3/W2: The main concerns are (i) accuracy, latency and footprint relative to EncT5, T5Enc and DirectParallel‑GP, (ii) the cost of converting an existing chain versus choosing a new training pipeline, and (iii) whether the source‑chain baseline preserves decoder prefix semantics. We address them in turn:

  1. Accuracy, latency, and footprint. We added the following five‑run EURLEX comparison with EncT5, T5Enc and DirectParallel‑GP.
MethodAvgF1 (mean ± SD)Median latency(ms)Params(M)Storage(MiB)
Chain2Parallel68.96 ± 0.5720.95109.7419.4
DirectParallel‑GP66.12 ± 2.5320.58109.7419.4
EncT557.19 ± 2.987.24119.12454.41
T5Enc32.57 ± 2.0410.23222.94850.46

From the table, we can see that Chain2Parallel achieves substantially higher accuracy with fewer parameters and a smaller storage footprint than EncT5/T5Enc, which offer lower latency. Compared with the same‑topology DirectParallel‑GP baseline, it achieves higher accuracy at essentially the same latency and footprint in this setting.

  1. Cost of Converting An Existing Chain vs. Choosing A New Training Pipeline. We report total measured training and conversion time from the shared GP checkpoint. The table gives means over five splits on the same RTX 4090D, with one conversion run per split.
DatasetChain2Parallel total (min)DirectParallel‑GP total (min)Conversion only (s)
EURLEX116.2422.3625.90
AAPD73.6516.7123.27
Reuters152.8920.8435.03

Chain2Parallel includes DA, order selection, CS and conversion (checkpoint loading, sentinel extraction, head fitting, export). DirectParallel‑GP includes encoder adaptation and readout training. Both reuse the GP checkpoint and avoid re‑running GP pretraining; Nguyen et al. report GP pretraining takes ~10 hours on an A100 (their Appendix A).

While the full DA+CS pipeline has higher training cost, pre‑trained chain checkpoints can be converted cheaply: conversion takes 23–35 s, delivering 50.9–214.5× faster inference and 55.7% fewer deployed parameters than the source chain without further encoder training. We have added this table to the Appendix.

  1. Decoder prefix semantics. The speedups reported in Table 3 are measured against the T5 Chain baseline, which preserves decoder‑prefix semantics. This baseline runs sentinel forward passes with decoder KV caching; YES/NO constraints are applied after full‑vocabulary projection at answer positions. Chain2Parallel eliminates the decoder and its serial autoregressive passes, while retaining label sentinels within the encoder input.
Reviewer_zKGpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose to convert a trained autoregressive T5 classifier chain into a parallel classifier after training, for few-shot multi-label text classification. The starting point is T5 Chain, which is trained in three stages. Chain2Parallel throws away the autoregressive decoder and freezes the specialized encoder. The encoder reads the document together with all label descriptions, and the method takes the hidden state at each label's sentinel token. A saved mapping puts these states back into the canonical label order. A lightweight shared MLP, trained with binary cross-entropy on the gold labels, then scores every label in one parallel pass.

Critical strengths
  • S1: As a clear and practical motivation, separating training-time dependency modeling from inference-time cost is a real deployment concern.
  • S2: The conversion only needs a frozen encoder, sentinel extraction, a fixed reordering and a small shared MLP. It is cheap to train and easy to reproduce.
  • S3: No loss of accuracy on EURLEX and AAPD, with lower variance across splits than T5 Chain.
  • S4: The authors connect the Reuters gain from sharing to labels with no positive training examples.
Critical weaknesses
  • W1: The novelty is limited as at its core the method is already the design of T5Enc and EncT5. The new part is the choice of starting checkpoint instead of a new learning principle.
  • W2: Without a key control like parallel BCE model from the domain-adapted checkpoint, we cannot tell whether the autoregressive chain objective matters or whether the domain-adaptation data exposure is doing the work.
  • W3: Total cost is not reported. Chain2Parallel requires the full GP →\rightarrow DA →\rightarrow CS chain pipeline and then head training. The paper reports no training compute and does not compare it with the direct-parallel alternatives.
  • W4: Label dependencies are not evaluated where they matter.
Questions for authors
  • Q1: When you compare Chain2Parallel against your own reproduced T5 Chain on Reuters, how much of the gain remains?
  • Q2: How sensitive are the results to the selected chain order?
3 comments
AuthorsOpenReview ↗

Thank the reviewer very much for all insightful comments.

Response to Reviewer zKGp

W1: We appreciate this comment and would like to clarify that our contribution goes beyond selecting a starting checkpoint. The following points address the concern:

  1. While our model builds upon the T5 architecture, the core contribution lies in a new functional decoupling: we separate chain‑based representation learning from sequential prediction.

  2. Unlike EncT5 and T5Enc, which construct label representations in a non‑autoregressive decoder using learned queries and fixed label tokens, respectively, our framework freezes the trained chain encoder, retains its label‑conditioned input, discards the entire decoder, and trains only a lightweight shared MLP on label‑aligned sentinel states. This yields an encoder‑only inference pipeline with no decoder passes or label‑specific binary heads and requires no further encoder updates.

  3. The distinction also has practical value in few‑shot classification: in our matched EURLEX comparison, Chain2Parallel achieves substantially higher AvgF1 than both EncT5 and T5Enc, with lower deployment overhead.

  4. The selection of the pre‑trained checkpoint is not the novelty itself; it enables us to empirically validate this decoupling.

  5. The central finding is that representations learned by a fully trained classifier chain can support effective parallel prediction after the sequential decoder is removed, without further encoder training.


W2: It raises one concern: whether the gains reflect the benefit of autoregressive chain training or merely exposure to target‑domain data. The following two points address this concern:

  1. The existing DirectParallel‑GP control already matches initialization and target‑domain data. Both routes start from the same GP checkpoint and use the same target‑domain examples (§4.2 and Appendix A); they differ in that one uses parallel BCE adaptation while the other uses autoregressive DA/CS followed by conversion. In Table 1, Chain2Parallel yields mean AvgF1 gains of 2.84/1.56 points on EURLEX/AAPD and achieves comparable performance on Reuters. The chain‑training route therefore shows a benefit without access to extra target‑domain examples.

  2. The proposed DA‑initialized control would test the training choice after DA. DA already trains the model to generate label answers autoregressively with shuffled label orders. A parallel model initialized from DA would therefore inherit target‑domain chain training and test whether to proceed with CS or switch to parallel adaptation at this stage. Our GP‑initialized comparison evaluates DA/CS as a complete chain‑training route, covering the target‑task training that produces the checkpoint we convert.


W3: Total Training Cost and Comparison with Direct‑Parallel Alternatives. We now report the total measured training and conversion time starting from the shared GP checkpoint. The table below reports mean wall‑clock times across five splits on the same RTX 4090D, with one conversion run per split.

DatasetChain2Parallel total (min)DirectParallel‑GP total (min)Conversion only (s)
EURLEX116.2422.3625.90
AAPD73.6516.7123.27
Reuters152.8920.8435.03

Here, the Chain2Parallel total includes DA, order selection, CS, and conversion; conversion covers checkpoint loading, sentinel extraction, head fitting, and export. The DirectParallel‑GP total covers encoder adaptation and readout training. Both methods reuse the same GP checkpoint and do not repeat GP pretraining. The original training cost of the GP checkpoint is reported separately: Nguyen et al. report approximately 10 hours on an A100 (Appendix A of their paper).

While the full “DA+CS” chain pipeline incurs higher training cost, the low conversion overhead enables practical reuse of pre‑trained chain checkpoints: conversion takes only 23–35 seconds, and the resulting model delivers 50.9–214.5× faster inference with 55.7% fewer deployed parameters than the source chain, without additional encoder training. We have added this table to the Appendix.


W4: Label Dependencies. We evaluate label‑dependency structure in §4.4 and Appendix C: the frozen representations used for parallel prediction preserve label co‑occurrence structure on EURLEX/AAPD. Specifically, we compare pairwise similarities of label‑state variation against training label co‑occurrence. After controlling for label‑specific class means and shared document components, the residual correlations reach 0.445 on EURLEX and 0.295 on AAPD. These values exceed all permutation‑based references across every split. Thus, cross‑label structure remains accessible to the parallel readout head after decoder removal.

AuthorsOpenReview ↗

Continue ...

Q1: Comparison between Chain2Parallel and Reproduced T5 Chain. The full +8.17 AvgF1 gain on Reuters is preserved when compared against our reproduced T5 Chain. This comparison is already reported in Table 1: AvgF1 increases from 71.2 to 79.3, with a paired 90% CI of [+6.26, +10.09] and consistent improvements across all five splits. The published 77.1 is an independent reference and is not used for the gain calculation.


Q2: Sensitivity of Selected Chain Order. Our evaluation focuses on converting an existing trained chain with its checkpoint and selected order fixed, rather than sweeping over alternative label orders. Nguyen et al. (2025, §4.2) studied T5 Chain’s sensitivity to label order and found that shuffling labels during domain adaptation reduces performance variation across chain orders. They report relative variation in AvgF1 below 3.5% across the tested orders.

Thanks for the response. As some concerns and novelty are still not fully addressed, I will keep my original rating.

Reviewer_GsWyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

When a classifier predicts label at a time, it can look at previous results for each prediction, but that can be slow in inference. This paper builds on top of a trained “T5 Chain” (Nguyen et al., 2025) and turns into a classifier model that predicts all labels in one pass, which they call, “Chain2Parallel”. The input gives each label a marker token. The method keeps the chain’s encoder unchanged and uses its output at each marker token as its feature. A “Shared MLP”, a smaller classifier, scores every label from these features, and the decoder is removed. The core claim of the paper is that chain training can teach the encoder model good features, removing the resultant slow decoder in test time deployment. They validate their method in 3 benchmarks (EURLEX, AAPD, Reuters) where the new model scores about the same as the chain on the first two and higher on Reuters. Chain2Parallel runs 51x to 214x faster with 56% fewer parameters. Similarly, an analysis of the encoder’s features is offered as an explanation of why the method works. However, some of the claims that matter for the central results are not statistically significant or statically underreported.

Critical strengths

Strengths ** The method is quite simple while keeping the chain’s accuracy at cheap. It keeps the chain’s encoder, adds one small classifier shared by all labels, and drops the decoder, so the 2m+2 decoding steps are gone. Table 3 shows that batch-one inference is faster and uses about half the memory, while AvgF1 stays at the chain’s level on EURLEX and AAPD.

** The sentinel experiment is clean, and the paper is honest about its limits. Table 2(a) changes only the label feature and keeps everything else fixed. Using the sentinel state instead of an average over all encoder states improves AvgF1 by 10 to 16 points on all three datasets, which shows clearly which part of the design matters. The paper also marks which baseline results in Table 1 come from different splits, and the authors are honest that the chain falls short of the published papers.

Critical weaknesses

** The main comparisons do not report confidence intervals, which makes it difficult to judge how strong the conclusions are. Appendix A mentions that the authors computed paired 90% confidence intervals across the five splits, but these are not shown. I am not sure why they were left out, especially since the differences behind the main claims are small. Compared with T5 Chain, Chain2Parallel achieves 69.0 vs. 68.7 AvgF1 on EURLEX and 54.7 vs. 54.7 on AAPD. The gains over DirectParallel-GP are also relatively small, at 2.9 and 1.5 points, respectively. Without the intervals, it is hard to judge how confidently we can say that the method matches T5 Chain or that chain training actually helps. This affects my confidence in the paper's conclusions and my overall rating.

** The paper's central argument in Section 1, that chain training is what makes the learned representations effective, is not fully supported by the results. On Reuters, the improvement over T5 Chain (71.2 to 79.3 AvgF1) is based on the authors' reproduced baseline, which is already 5.9 points below the published result of 77.1 from Nguyen et al. (2025), the paper this work builds on. At the same time, DirectParallel-GP, which does not use chain training, achieves 79.6 AvgF1. This makes it difficult to see how much of the improvement really comes from chain training. It also conflicts with the statement in the introduction that the DirectParallel baselines are “obviously inferior to T5 Chain.”

Questions for authors

Please see the weaknesses.

1 comment
AuthorsOpenReview ↗

Thank the reviewer very much for all insightful comments.

Response to Reviewer GsWy

W1: Confidence intervals. Thank you for pointing out the missing confidence intervals. The table below reports paired AvgF1 differences and two‑sided 90% Student’s t‑intervals across five matched splits (Chain2Parallel minus reference). This table has been added to the Appendix.

Datasetvs. local T5 Chainvs. DirectParallel‑GP
EURLEX+0.22 [−0.51, +0.95]+2.84 [+0.72, +4.96]
AAPD−0.01 [−0.76, +0.75]+1.56 [−1.18, +4.29]
Reuters+8.17 [+6.26, +10.09]−0.26 [−2.35, +1.82]

From this table, Chain2Parallel matches the source chain's aggregate performance on EURLEX/AAPD and improves it on Reuters, consistent with our main conclusion. Its advantage over T5 Chain is the substantial reduction of inference latency, memory consumption, and model footprint while preserving classification performance.

Against DirectParallel‑GP, the mean AvgF1 gains of 2.84/1.56 on EURLEX/AAPD are non‑trivial for the few‑shot MLTC task, while performance on Reuters is comparable.


W2: Thanks for this insightful comment. It raises three related concerns: (i) the benefit of task‑specific chain training, (ii) the scope of our introductory comparison, and (iii) the Reuters reproduction gap. We address them in turn:

  1. The benefit of task‑specific chain training can be validated. Starting from the same GP checkpoint, we compare readouts with and without DA/CS under frozen encoders, using the same shared MLP, inputs, and model selection protocol across three matched splits per dataset (AvgF1, mean ± sample SD).
DatasetWithout DA/CSWith DA/CSGain
EURLEX50.83 ± 4.1968.72 ± 0.66+17.89
AAPD33.72 ± 3.1254.46 ± 0.14+20.74
Reuters67.54 ± 1.5279.11 ± 2.06+11.57

As shown in this table, task‑specific chain training substantially improves performance on all three datasets. With the encoder frozen during readout training and an identical parallel readout setting, these gains support our statement that “task‑specific few‑shot chain training is helpful for classification performance.” The comparison against DirectParallel‑GP addresses a distinct question: the relative performance of these two adaptation routes. In Table 1 of our paper, the strong Reuters result of DirectParallel‑GP shows that parallel adaptation is also effective on this dataset; the sparse label co‑occurrence and long‑tailed label distribution of Reuters may help explain why it remains competitive on this benchmark.

  1. The scope of the introductory comparison. We revise the introductory comparison to state its scope explicitly, replacing “They however are obviously inferior…” with “They however are inferior to T5 Chain and our method in the majority of cases”. Recall that in Table 1 there are 4 metrics, 2 DirectParallel baselines, and 3 datasets, yielding a total of 24 (4×2×3) cases; T5 Chain performs better on 20 out of 24.

  2. The Reuters reproduction gap. The exact random seeds and selected learning rates for the published main table are not provided in the T5 Chain paper or its released code. We explored multiple hyperparameter configurations, yet these alternatives (Appendix A) failed to recover the published Reuters result. Although this specific result remains unrecovered, it does not undermine the main conclusion of our paper (i.e., Chain2Parallel matches or improves upon the original chain’s classification performance while substantially reducing inference latency, memory consumption, and model footprint).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.