- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SoftPaths, a two-stage method that gives a decoder several different latent conditions for the same question. A frozen LLaMA-3.2-1B assistant produces hidden states, and four path-specific routers mix a shared pool of projection experts to turn them into four soft-prompt sequences for a LLaMA-3.1-8B base model. After a short rationale-supervised LoRA warm-up, the projector is trained with an answer-only loss plus a contrastive loss that pushes the paths apart. The projector is then frozen, and a new LoRA adapter is trained with GRPO. Paths are cycled across steps, and the reward only credits correct answers and penalizes correct responses that are longer than the group's average correct response. At test time the model either decodes from one path, or decodes once from each of several paths and takes a majority vote. On GSM8K, ASDiv-Aug, MATH, CommonsenseQA and StrategyQA, the paper reports the best average accuracy at M=1 and M=4 with lower visible-token usage than the matched CoT baselines. It also includes analyses of candidate scaling, path complementarity and ablations.
Critical strengths
-
Good results at matched candidate counts. SoftPaths has the highest five-benchmark average at both budgets: 72.20 at M=1 (strongest external baseline: Self-SoftCoT, 70.79) and 76.51 at M=4 (strongest external baseline: Assist-CoT-SC, 72.69). The visible-token metric includes all four responses at M=4. The paper is also upfront that fewer visible tokens does not mean lower end-to-end latency.
-
The analysis goes beyond a main table. The GSM8K ablations show drops when the contrastive loss, the MoE projector or the length reward is removed. Figure 5 breaks the voting gain down into mean-path accuracy, Oracle@4, mixed correctness and answer disagreement. The correctness-gated reward is a sensible choice, since a short but wrong answer gets nothing.
Critical weaknesses
-
The comparisons do not isolate the contribution of latent paths beyond decoder RL. SoftPaths fine-tunes the base model (a warm-up LoRA, then GRPO with a correctness-gated length reward). The explicit CoT baselines are training-free, while SoftCoT trains its projector with the base decoder frozen. Self-SoftCoT is trained with RL, but only through the projector, and it is reported only at M=1. There, it beats SoftPaths on ASDiv-Aug (92.35 vs 89.88) and CSQA (75.18 vs 75.02), and its ASDiv-Aug score is above SoftPaths' own M=4 result (92.00). The w/o-GRPO row shows what GRPO adds to SoftPaths. It does not show what the latent paths add on top of GRPO. A no-latent model trained with the same warm-up, LoRA-GRPO and reward is the missing control, and a good part of the accuracy gain and the token reduction could come from the decoder training and the length reward alone. This matters because latent-path construction is a central contribution, yet the current controls do not separate its benefit from the combined decoder-training recipe. The tables provide no uncertainty estimates, and Appendix B.2 does not specify the number of decoding seeds or independent training runs. Small gains, such as the 0.23-point GSM8K improvement from GRPO at M=1 and the 0.68-point length-reward ablation, therefore cannot be confidently distinguished from variation.
-
The claim of functionally diverse paths is not sufficiently isolated from sampling diversity. The contrastive loss encourages separation in representation space, but does not by itself establish different reasoning strategies or decoder-relevant information. Such separation could reflect path-specific signals with little effect on decoder behavior. Figure 1 is an illustration and Appendix C states that its case study is not causal evidence. Figure 5 shows disagreement and a voting gain, but lacks a within-model sampling control: the same metrics for four samples from one fixed path of the same checkpoint are needed to distinguish path-conditioned complementarity from stochastic decoding. SoftCoT++ already builds multiple latent thoughts with specialized initial tokens and contrastive learning. The advance over it therefore needs to be established through shared-expert path construction and latent-conditioned policy optimization, rather than representation diversity alone. The shared-linear-projector ablation does not distinguish the benefit of MoE composition from that of having multiple path-specific mappings. Independent projectors of comparable size, trained with the same losses, would provide a more informative control. This matters because functional path diversity is a central claim, and the current controls do not establish that distinct latent conditions help beyond sampling from one condition.
Questions for authors
- For the same trained SoftPaths checkpoint, how do four different paths compare with four samples from one fixed path, under matched decoding and voting? Please report accuracy and the Figure 5 metrics. Which path is used for M=1, and how does its GSM8K accuracy of 84.31% relate to Figure 5a's mean-path accuracy of 84.51%?
- How does a no-latent model perform with the same warm-up, LoRA-GRPO and correctness-gated length reward? If feasible, a four-candidate Self-SoftCoT result would also help the matched-budget comparison.
- What are the expert and router architectures? How does the shared-expert projector compare with independent path projectors of similar size trained with the same answer and diversity losses?
- How do you transition from eight latent vectors during projector training to one in Stage II and inference? What happens to the warm-up LoRA when the fresh GRPO adapter is introduced? Please also give the training data and splits, the rationale source, the reward and diversity coefficients, and how ties and invalid outputs are handled in voting.
- How many decoding seeds and independent training runs are behind the reported numbers? Can you provide uncertainty estimates for the main comparisons and key ablations, say which baseline numbers were reproduced and which were taken from prior work, and say whether code will be released?
3 comments
[2/3]
(3)Reasoning Diversity.
Table R3. Lexical and semantic diversity of visible reasoning (M=4).
| Dataset | Method | Distinct-2 ↑ | Length-matched D2 ↑ | Semantic distance ↑ | Reasoning tokens/candidate ↓ |
|---|---|---|---|---|---|
| ASDiv-Aug | SoftPaths | 0.4868 | 0.5697 | 0.1539 | 60.3 |
| SoftCoT-SC | 0.4458 | 0.4839 | 0.1075 | 100.9 | |
| SoftCoT++ | 0.4802 | 0.5381 | 0.1081 | 90.2 | |
| Assist-CoT-SC | 0.4145 | 0.4367 | 0.0824 | 98.7 | |
| CommonsenseQA | SoftPaths | 0.7219 | 0.7546 | 0.2480 | 46.9 |
| SoftCoT-SC | 0.7161 | 0.7489 | 0.2461 | 109.9 | |
| SoftCoT++ | 0.6616 | 0.7349 | 0.2293 | 111.8 | |
| Assist-CoT-SC | 0.6576 | 0.6842 | 0.1864 | 164.8 |
Metrics exclude final answers and fixed templates. Distinct-2 measures unique-bigram proportion; length-matched D2 truncates each question's four rationales to the shortest common length; semantic distance averages pairwise MiniLM cosine distances; reasoning tokens/candidate measures average rationale length.
SoftPaths achieves the highest lexical and semantic diversity with shorter reasoning traces, providing behavioral evidence that latent-path conditioning induces more varied visible reasoning outputs.
Together, Tables R1–R3 support observable decoding diversity and predictive complementarity, rather than merely geometric separation of latent representations.
M=1 evaluation clarification (about 84.31% in Table 1 and 84.51% in Figure 5). Table 1 uses balanced random path assignment, yielding 84.31% (1,112/1,319). Figure 5(a) averages four paths evaluated separately on the full test set, yielding 84.51%. We will clarify both protocols in the revision.
Q2 / W1. Are the gains merely attributable to decoder-side GRPO?
No-latent GRPO baseline. To isolate the contribution of latent paths, we introduce LoRA-GRPO, a no-latent baseline trained using SFT followed by GRPO, with matched backbone, training budget, reward, and evaluation settings.
Table R4. Comparison with the no-latent GRPO baseline (ASDiv-Aug, M=4).
| Method | Vote@4 (%) ↑ | Oracle@4 (%) ↑ | Visible tokens/question ↓ |
|---|---|---|---|
| SoftPaths | 92.00 | 96.05 | 333.2 |
| LoRA-GRPO | 90.75 | 95.18 | 366.4 |
SoftPaths outperforms matched decoder-only GRPO, providing evidence that learned latent paths contribute beyond the decoder-training recipe.
Self-SoftCoT comparison. We acknowledge that Self-SoftCoT performs better on some individual M=1 benchmarks. However, SoftPaths achieves higher five-benchmark average M=1 accuracy (72.20% vs. 70.79%). At M=4, SoftPaths reaches 76.51%, compared with 72.69% for Assist-CoT-SC. We will clarify the candidate-budget differences.
Q3 / W2. Does shared-expert routing outperform independent projectors?
SoftPaths uses four shared 2048→4096 linear experts and path-specific routing (2048→16), with Gumbel-Softmax during training. We compare it with four independent linear projectors under matched supervision and training budgets.
Table R5. Shared-expert routing versus independent projectors (GSM8K, Stage I, M=4).
| Method | Parameters ↓ | Vote@4 (%) ↑ |
|---|---|---|
| SoftPaths | 33,587,200 | 89.76 ± 0.35 |
| Independent projectors | 33,554,432 | 87.96 ± 0.42 |
With nearly identical parameter counts, shared-expert routing achieves higher voting accuracy and correct-answer coverage, supporting the benefit of SoftPaths.
[3/3]
Q4. How are the two stages connected, and what are the implementation details?
Latent-slot transition. Our implementation follows the prefix-length adjustment used in the official SoftCoT/SoftCoT++ code, which uses 32 soft-token positions during training and 4 during evaluation. Importantly, SoftPaths generates only one latent vector per path, which is repeated across eight positions in Stage I and inserted into a single position in Stage II. Thus, no independent latent vectors are discarded. We evaluate the same Stage-I SoftPaths-SFT checkpoint before GRPO, with the warm-up LoRA removed, using identical decoding settings.
Table R6. Effect of the 8-to-1 latent-slot transition.
| Dataset | 8→1 Accuracy (%) | 8→8 Accuracy (%) |
|---|---|---|
| ASDiv-Aug | 87.38 | 87.48 |
| CommonsenseQA | 75.10 | 75.10 |
The nearly identical accuracy indicates that reducing the repeated latent prefix to one position does not materially affect answer-level performance in the evaluated settings.
About warm-up LoRA lifecycle. The warm-up Stage-I LoRA is trained with supervised reference responses and then frozen to provide a stable decoder for projector training. Before Stage II, it is removed, and a fresh LoRA is initialized for GRPO, avoiding the formatting bias learned during supervised warm-up.
Data and supervision. Both stages use the same task-specific training partitions.
Table R7. Training and evaluation partitions.
| Dataset | Stage I / Stage II training | Evaluation |
|---|---|---|
| GSM8K | Train (7,473) | Test (1,319) |
| ASDiv-Aug | Train (4,183) | Test (1,038) |
| MATH | Train (7,348) | MATH500 (500) |
| CommonsenseQA | Train (9,741) | Labeled Validation (1,221) |
| StrategyQA | First 80% (1,832) | Remaining 20% (458) |
Training details. Both stages use the training partitions above, with no exact normalized question overlap with evaluation sets. Stage-I contrastive weight/temperature/dropout: 0.1/0.05/0.1. Stage-II GRPO uses G=8, a formatting coefficient of 0.1, and a correctness-gated length penalty of 0.05.
We will provide the complete implementation details in the revision.
Q5 / W1. Are the improvements reliable and reproducible?
Length-reward ablation. We reevaluate the GSM8K length-reward ablation using fixed checkpoints and three decoding seeds.
Table R8. Three-seed GSM8K length-reward ablation (M=4).
| Decoding seed | Full SoftPaths Vote@4 (%) ↑ | No length reward Vote@4 (%) ↑ | Gain (pp) | Full tokens/question ↓ | No length reward tokens/question ↓ |
|---|---|---|---|---|---|
| Mean ± SD | 90.37 ± 0.70 | 89.11 ± 0.94 | +1.26 ± 0.39 | 782.07 ± 11.93 | 815.23 ± 15.10 |
The correctness-gated length reward improves voting accuracy and reduces visible tokens across all three decoding seeds, supporting its effectiveness beyond a single evaluation run.
We additionally report three-seed results on ASDiv-Aug and CommonsenseQA.
Table R9. Multi-seed accuracy and visible-token efficiency (M=4, mean ± sample SD).
| Dataset | Method | Vote@4 (%) ↑ | Visible tokens/question ↓ |
|---|---|---|---|
| ASDiv-Aug | SoftPaths | 92.00 ± 0.48 | 323.72 ± 1.99 |
| SoftCoT-SC | 89.76 ± 1.07 | 473.45 ± 8.69 | |
| LoRA-GRPO | 90.75 ± 0.24 | 366.57 ± 1.66 | |
| CommonsenseQA | SoftPaths | 77.81 ± 0.64 | 249.82 ± 1.11 |
| SoftCoT-SC | 66.53 ± 0.70 | 675.05 ± 10.84 | |
| LoRA-GRPO | 76.74 ± 0.36 | 277.32 ± 1.01 |
SoftPaths achieves the highest mean voting accuracy and lowest visible-token consumption on both benchmarks, providing additional evidence of its advantages over repeated sampling and no-latent decoder optimization.
Reproducibility. Coconut and Self-SoftCoT results are taken from their original papers; other external baselines are locally reproduced using official implementations. We will clarify baseline sources and release our code, configurations, reward functions, and voting implementation upon acceptance.
We appreciate the reviewer's suggestions and will incorporate these additional results and clarifications into the revised manuscript.
[1/3]
We thank the reviewer for the constructive feedback. We conducted additional controlled experiments and provide clarifications below.
Q1 / W2. Do multiple latent paths provide benefits beyond stochastic sampling?
(1)Controlled sampling comparison. We directly compare four distinct latent paths with four samples from a single fixed path, using the same SoftPaths checkpoint, prompts, decoding settings, and candidate budget. The single-path results are averaged over all four path choices.
Table R1. Multi-path versus single-path sampling (M=4).
These controlled results demonstrate that learned latent paths provide additional predictive complementarity beyond repeated stochastic sampling from a single latent condition.
(2)Predictive complementarity. We further compare predictive complementarity and visible reasoning diversity with representative baselines.
Table R2. Predictive complementarity across methods (M=4).
MP: mean candidate accuracy; Vote@4: voting accuracy; Oracle@4: at least one correct candidate; MC: mixed correctness (1–3 correct candidates); AD: at least two distinct normalized answers (unparseable outputs form one category).
SoftPaths achieves the highest voting accuracy and correct-answer coverage on both benchmarks, supporting the effectiveness of latent-path complementarity across different reasoning tasks.