- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper converts a T5 classifier chain into a parallel classifier by freezing its encoder and training a shared MLP on label-aligned sentinel states. Across five splits, AvgF1 is maintained on EURLEX and AAPD and improves over the local chain on Reuters. Removing the decoder reduces deployed parameters by 55.7% and gives 50.9–214.5× faster batch-one inference under the reported implementation.
Critical strengths
- The conversion offers a useful deployment option for existing CHAIN checkpoints. It preserves aggregate performance on EURLEX and AAPD while substantially reducing inference cost. Both timing paths include fresh encoder passes, and the chain baseline uses decoder caching.
- The ablations clarify what matters, i.e., label-aligned sentinel states consistently outperform pooled features, while sharing and MLP gains vary by dataset. The representation analysis also controls for shared document and class-mean components rather than relying on raw correlations alone.
Critical weaknesses
- The benefit of task-specific chain training is not fully isolated. Chain2Parallel exceeds DirectParallel-GP by 2.9/1.5 AvgF1 points on EURLEX/AAPD but trails by 0.3 on Reuters, contradicting the introduction’s blanket claim. Frozen GP/DA/CS readouts would clarify the later stages’ contributions. The Reuters gain also needs qualification because the local source chain underperforms the published result.
- The efficiency comparison mainly supports converting an existing chain. For a new training pipeline, DirectParallel-GP offers the same deployment topology and competitive accuracy. Matched deployment and training-cost comparisons, together with an EncT5/T5Enc comparison, can better establish the method’s practical advantage.
Questions for authors
- Please report split-level paired AvgF1 differences and confidence intervals against both the local chain and DirectParallel-GP. Does the Reuters improvement persist with stronger source checkpoints? Please also revise the introduction’s claim about the direct baselines.
- Please compare identical shared heads trained on frozen GP, DA, and CS states, with matched inputs and model selection. Please include feature-extraction and head-training time.
- Can you report latency and footprint alongside DirectParallel-GP and clarify when chain training is worth its additional cost? A matched EncT5/T5Enc comparison would also help. Any optimized-chain comparison should preserve the sentinel prefixes required by subsequent decisions.
2 comments
Continue ...
Q3/W2: The main concerns are (i) accuracy, latency and footprint relative to EncT5, T5Enc and DirectParallel‑GP, (ii) the cost of converting an existing chain versus choosing a new training pipeline, and (iii) whether the source‑chain baseline preserves decoder prefix semantics. We address them in turn:
- Accuracy, latency, and footprint. We added the following five‑run EURLEX comparison with EncT5, T5Enc and DirectParallel‑GP.
| Method | AvgF1 (mean ± SD) | Median latency(ms) | Params(M) | Storage(MiB) |
|---|---|---|---|---|
| Chain2Parallel | 68.96 ± 0.57 | 20.95 | 109.7 | 419.4 |
| DirectParallel‑GP | 66.12 ± 2.53 | 20.58 | 109.7 | 419.4 |
| EncT5 | 57.19 ± 2.98 | 7.24 | 119.12 | 454.41 |
| T5Enc | 32.57 ± 2.04 | 10.23 | 222.94 | 850.46 |
From the table, we can see that Chain2Parallel achieves substantially higher accuracy with fewer parameters and a smaller storage footprint than EncT5/T5Enc, which offer lower latency. Compared with the same‑topology DirectParallel‑GP baseline, it achieves higher accuracy at essentially the same latency and footprint in this setting.
- Cost of Converting An Existing Chain vs. Choosing A New Training Pipeline. We report total measured training and conversion time from the shared GP checkpoint. The table gives means over five splits on the same RTX 4090D, with one conversion run per split.
| Dataset | Chain2Parallel total (min) | DirectParallel‑GP total (min) | Conversion only (s) |
|---|---|---|---|
| EURLEX | 116.24 | 22.36 | 25.90 |
| AAPD | 73.65 | 16.71 | 23.27 |
| Reuters | 152.89 | 20.84 | 35.03 |
Chain2Parallel includes DA, order selection, CS and conversion (checkpoint loading, sentinel extraction, head fitting, export). DirectParallel‑GP includes encoder adaptation and readout training. Both reuse the GP checkpoint and avoid re‑running GP pretraining; Nguyen et al. report GP pretraining takes ~10 hours on an A100 (their Appendix A).
While the full DA+CS pipeline has higher training cost, pre‑trained chain checkpoints can be converted cheaply: conversion takes 23–35 s, delivering 50.9–214.5× faster inference and 55.7% fewer deployed parameters than the source chain without further encoder training. We have added this table to the Appendix.
- Decoder prefix semantics. The speedups reported in Table 3 are measured against the T5 Chain baseline, which preserves decoder‑prefix semantics. This baseline runs sentinel forward passes with decoder KV caching; YES/NO constraints are applied after full‑vocabulary projection at answer positions. Chain2Parallel eliminates the decoder and its serial autoregressive passes, while retaining label sentinels within the encoder input.
Thank the reviewer very much for all insightful comments.
Response to Reviewer dW12
Q1: This comment asks for (i) paired results, (ii) a correction to the introductory comparison, and (iii) clarification of whether the Reuters gain extends to stronger source checkpoints. We address them in turn:
These results support our main finding: conversion preserves aggregate chain performance on EURLEX/AAPD. For Reuters, Chain2Parallel outperforms our reproduced T5 Chain across all five splits. These paired results have been added to the Appendix.
Introductory Wording. We revise the introductory comparison to state its scope explicitly, replacing “They however are obviously inferior…” with “They however are inferior to T5 Chain and our method in the majority of cases”. Recall that in Table 1 there are 4 metrics, 2 DirectParallel baselines, and 3 datasets, yielding a total of 24 (4 × 2 × 3) cases; T5 Chain performs better on 20 out of 24.
Scope of the Reuters Gain. The large Reuters improvement of Chain2Parallel is observed against our locally trained T5 Chain checkpoint, which failed to reproduce the original published performance. Whether this large gain persists when converting from the stronger official T5 Chain checkpoint remains unclear, as the learning rate and random seed settings are not available from their paper and released code. We have revised Section 4.2 (Main Results) to explicitly state that the Reuters gain is relative to its locally trained source T5 Chain. The accuracy retention and deployment savings on EURLEX/AAPD establish our method's practical value independently of the Reuters improvement.
Q2/W1: The concerns are (i) the incremental benefit of DA/CS for the frozen readout, and (ii) the cost of extracting features and fitting the head. We address these separately:
This table shows that removing DA/CS substantially lowers performance on every dataset, supporting the incremental value of task‑specific chain training. The comparison of adaptation routes in our paper’s Table 1 is a separate result: Chain2Parallel has higher mean AvgF1 on EURLEX/AAPD, while DirectParallel‑GP is slightly ahead on Reuters. Its strong Reuters result shows that parallel adaptation is also effective there; Reuters’ sparse co‑occurrence and long‑tailed labels may help explain why it remains competitive. We have added this table to the Appendix.