acceptodds
Under review as a conference paper at ICLR 2027

Source or Prefix? Retaining What Helps the Target in Autoregressive 3D Medical Image Translation

Abstract

Translating between 3D medical imaging modalities requires preserving patient anatomy while recovering target-specific properties. A source-conditioned blockwise autoregressive model uses the source as a fixed anatomical anchor and earlier blocks as an evolving target prefix, and the difference between predictions with and without the prefix defines the prefix correction. The balance between source and prefix evidence varies across regions, while the magnitude of a correction's prediction change (effect) does not necessarily reveal whether it helps the true target (utility). In this work, we propose Source-Prefix Influence-aware Residual Autoregression (SPIRA), which learns how much of each prefix correction to retain across regions. During training, SPIRA finds the largest removable portion of each correction without increasing target loss at the current prediction states and trains a lightweight gate to reproduce the resulting predictive distribution, thus learning how to balance source and prefix during generation. Extensive experiments across multiple modalities and diverse 3D medical translation tasks demonstrate consistent gains in translation quality. Mechanism analyses show that similar prefix effects can have different utilities depending on target alignment, while SPIRA better matches oracle decisions and achieves larger gains when more prefix correction can be safely retracted.

Primary area:
generative models
Keywords:
Medical image synthesis, Autoregressive generation, Generative models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KM3rOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper revisits this question that "source image condition + autoregressive generation of target prefix", the prediction correction brought by the target prefix is regarded as a controllable and partially reversible correction. Their model learns to retain only the parts that are useful for the real target in each target region/token, and withdraw the harmful or unnecessary parts, thereby improving the quality of autoregressive generation. They claimed that similar prefix effects can have different utilities depending on target alignment, while SPIRA better matches oracle decisions and achieves larger gains when more prefix correction can be safely retracted. Then they use a lightweight MLP gate to learn the withdrawal amount through predictive-action distillation (KL matching of the oracle distribution) during test stage.

Critical strengths

They observed that predictive effect alone cannot determine the appropriate source–prefix balance, or equivalently how much of each prefix correction to retain. They claimed the reason is that the disalignment of the predictive effect and the target utility. They try to solve this phenomenon using eqs 4, 5 and 6.

Critical weaknesses

During the test, they use a lightweight MLP gate to predict the retraction. However, this paper do not clearly reveal the relation between the alignment and the learned boundary of gate.

Questions for authors

(1) Why they choose the MLP gate distilllation technique rather than others? Why are those features being input? What is the evidence? (2) This paper seems to be presenting as a speech draft, is it good?

2 comments
AuthorsOpenReview ↗

Response to Reviewer KM3r (Part 1/2)

We are grateful to the reviewer for recognizing our central observation that predictive effect alone cannot determine the appropriate source-prefix balance, and for highlighting the importance of target alignment. Building on these observations, we clarify the relationship between target alignment and the learned gate, summarize the existing experimental evidence supporting our gate design, and address the presentation concerns.

Summary of Key Concerns and Main Responses

Key ConcernsMain Responses
Alignment and learned gate (W1)Clarified how target alignment determines oracle decisions and how the learned gate approximates them, supported by existing analyses in Figs. 5, 6(b), and 10(a).
Gate design and distillation (Q1)Explained the choices of lightweight MLP, predictive-distribution distillation, and complementary input features, supported by existing ablations in Tables 3, 15, and 23.
Scientific presentation (Q2)Clarified the presentation choices for readability and welcomed further suggestions.

W1: Relationship between target alignment and the learned gate.

Thank you for highlighting the need to clarify this relationship. We clarify that target alignment determines the oracle's retraction boundary during training, while the learned gate approximates the oracle's behavior without accessing GT at inference. Specifically, Sec. 3.3, Eqs. (5)-(6) establish that target alignment and predictive effect jointly determine the maximal zero-regret retraction, as further illustrated in Sec. 4.4, Fig. 5. The gate then learns to approximate the resulting oracle predictive distribution using GT-free features.

Moreover, we clarify that we have provided two empirical analyses in our manuscript supporting this connection:

  • Oracle decision predictability (Appendix E.2, Fig. 10(a), page 29): The complete features outperform confidence- or effect-only features in predicting the three alignment-dependent oracle regimes.

  • Learned gate agreement (Sec. 4.5, Fig. 6(b), page 8): On identical generated-prefix states, SPIRA achieves a lower oracle-distribution KL divergence than the effect-only gate.

Overall, our analyses support the gate’s ability to approximate alignment-dependent oracle decisions from GT-free features, enabling adaptive source-prefix balancing without GT information at inference.


Q1: Justification of MLP gate.

Thank you for this thoughtful advice. Our current manuscript already provides experimental evidence and analyses supporting these design choices, which we clarify below.

  • Why a lightweight MLP? We choose a lightweight MLP because it can capture nonlinear interactions among source, correction, and endpoint features with minimal additional complexity. As detailed in Appendix B.3 (page 17), our gate uses a 128-dimensional hidden layer and introduces only 0.13M additional parameters (Appendix E.3, Table 23, page 31). Its effectiveness is further supported by the oracle-agreement analysis in Sec. 4.5, Fig. 6(b).

  • Why predictive-distribution distillation? We adopt distribution matching because our goal is to reproduce the oracle's predictions rather than its exact retraction scalar. The same scalar error can have different effects on the predicted distribution. In Sec. 4.3, Table 3 (page 6-7), SPIRA outperforms direct s∗s^* regression. Similarly, Appendix D.6, Table 15 (page 25) shows that SPIRA outperforms direct target-NLL optimization with the same gate inputs and architecture, supporting our choice of supervision.

  • Why these input features? We select these features because they provide complementary, GT-free information for retraction decisions, where uSu^S captures source evidence, uru^r represents the prefix-induced correction, and (w,e)(w,e) summarize differences between endpoint predictions (Appendix B.3, page 17). The ablations in Appendix D.6, Table 15 (page 25) indicate that removing or replacing these features consistently degrades performance, confirming their complementary contributions.

To sum up, our analyses support the lightweight gate design and show that the proposed features and distribution matching enable more effective retraction decisions than the evaluated alternatives.

AuthorsOpenReview ↗

Response to Reviewer KM3r (Part 2/2)

Q2: Presentation.

We thank the reviewer for this comment. We would like to clarify that our question-style subheadings and highlighted boxes were intentionally designed for two reasons:

  • Readability and clarity: These elements help readers follow the paper's motivation, methodology, and key findings, rather than serving as a speech-style presentation.

  • Visual organization: Their typography, color scheme, and visual hierarchy were carefully designed to maintain a clean, professional academic presentation while improving the organization and accessibility of the content.

We believe these design choices are compatible with scientific writing conventions.

Nevertheless, we understand the reviewer's concern and are open to refining the presentation to better align with conventional scientific writing, including reducing rhetorical questions or redundant takeaway boxes where appropriate. We would greatly appreciate any specific suggestions on sections that the reviewer finds overly presentation-oriented, and we would be happy to address them.


We are also grateful to the reviewer for the thoughtful comments and valuable suggestions. We hope these clarifications and existing experimental evidence adequately address the concerns. We warmly welcome any further questions, suggestions, or discussion to help us further strengthen our work.

Reviewer_jjxpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies source-conditioned autoregressive (AR) translation for 3D medical images. It observes that during block-wise AR generation, the evolving target prefix can be helpful in some regions but harmful in others, and that predictive effect alone—how much a prefix correction changes the prediction—does not determine whether the correction is useful for the true target. The authors propose SPIRA (Source-Prefix Influence-aware Residual Autoregression), which treats the target prefix as a controllable correction and learns a region-wise retraction amount. The key technical contribution is an exact decomposition of target-aware retraction regret into a predictive-effect term and a target-alignment term. During training, a ground-truth oracle finds the maximal zero-regret retraction, and a lightweight correction-aware gate distills the resulting predictive distribution. Experiments cover eight translation tasks on three datasets (BraTS 2024, SynthRAD 2025, AutoPET), with ablations on retraction granularity, gate inputs, supervision strategies, and inference efficiency.

Critical strengths

Strength 1: The retraction-regret decomposition is a clean and useful conceptual contribution. The paper shows that the change in target loss after retracting a prefix correction can be written exactly as predictive effect plus target alignment, and that effect alone cannot determine safe retraction. This is a simple but non-obvious observation that reframes the source-prefix balance problem in a principled way, and it explains why methods based only on predictive uncertainty or effect may fail. This is the strongest reason to consider the paper.

Strength 2: The empirical evaluation is broad and includes informative ablations. The framework is tested on eight translation tasks across three datasets, with comparisons against standard AR, full-prefix AR, global retraction, position-only gates, effect-only gates, and direct s∗s^* regression. The granularity ablation (block-wise vs. sub-block-wise vs. token-wise) and the gate-input ablation provide useful evidence about where the gains come from, rather than only reporting a single headline number.

Critical weaknesses

Weakness 1: The provided manuscript is materially corrupted, with multiple pages containing only repeated indices and several figures or sections unreadable. Page 1 consists entirely of line numbers, Page 3 and Page 5 are largely repeated indices, Figure 1’s caption begins with “igure 1,” and parts of the method and appendix appear truncated. This is not a cosmetic issue: the paper’s central claims depend on the retraction-regret decomposition, the oracle construction, and the gate supervision, and several of these sections are either missing or unreadable in the submitted file. Without a complete and readable manuscript, the technical correctness of the decomposition and the fairness of the experimental comparisons cannot be fully assessed. For a methods paper, this is a serious evaluability failure and supports rejection in the current form.

Weakness 2: The empirical comparison does not cleanly isolate the benefit of selective retraction from the benefit of dual-endpoint training itself. SPIRA trains a shared AR predictor with both full-prefix and source-only losses, and then adds a gate on top. The paper compares SPIRA against Full-prefix AR, which uses the same dual-endpoint predictor but disables retraction at inference. However, the standard AR baseline is trained separately without the source-only loss. The paper does not report a comparison between standard AR and a dual-endpoint model with a fixed global retraction tuned to match compute, nor does it fully separate the gain from the additional source-only training signal from the gain due to selective retraction. Since the paper’s central claim is that selective retraction is the key mechanism, this confound weakens the attribution. A cleaner experiment would train a dual-endpoint model with a single global retraction scalar and compare it against SPIRA under the same training budget; the paper mentions such a variant in the appendix, but the main text does not make the comparison sufficiently prominent.

Weakness 3: The oracle and gate are defined on teacher-forced GT prefixes, while inference operates on generated prefixes, and the paper does not adequately characterize the resulting distribution shift. The maximal zero-regret oracle uses the ground-truth target to score retraction actions under teacher forcing. The gate is then trained to distill the oracle’s predictive distribution. At inference, however, the prefix consists of generated blocks, which may contain errors. The paper acknowledges this train-test mismatch as a limitation, but the empirical analysis of generated-prefix states (e.g., the local regret audit in Table 18) is relatively limited and does not fully establish that the gate remains reliable when the prefix distribution shifts. This is consequential because the method’s core claim is that it improves free-running generation, not just teacher-forced prediction.

Questions for authors
  1. The provided manuscript has corrupted pages and unreadable figures. Can you provide a complete, readable version? Without the full method section and the main quantitative tables, the paper’s central claims cannot be verified.

  2. Can you provide a direct comparison between standard AR, a dual-endpoint model with a tuned global retraction scalar, and SPIRA under the same training budget? This would isolate how much of the gain comes from the additional source-only training signal versus selective token-wise retraction.

  3. The oracle is defined under teacher forcing with GT prefixes. How does the gate’s behavior change when evaluated on generated prefixes, and can you report the positive-regret rate and regret magnitude separately for early, middle, and late blocks to characterize the distribution shift?

  4. In Equation (5), the regret decomposition is stated as Ri,y(s)=Ei(s)+sAiR_{i,y}(s)=E_i(s)+sA_i. Can you provide the full derivation in the main text, including the definition of AiA_i and the convexity argument used for the oracle? This is central to the paper’s theoretical contribution.

  5. The gate is supervised by distilling the oracle predictive distribution rather than regressing s∗s^*. The ablation shows that direct s∗s^* regression underperforms, but can you explain why distribution matching is preferable beyond the empirical result? A more explicit justification would strengthen the methodological contribution.

  6. The method relies on paired source-target data for the flow-matching-style training and oracle construction. How sensitive is SPIRA to pairing quality, and can it be extended to unpaired or weakly paired settings?

  7. The inference cost analysis shows a modest latency increase over Full-prefix AR, but the training cost of dual-endpoint learning is substantially higher. Can you report the total training time and GPU memory relative to standard AR, and clarify whether the gains justify the additional training cost in practice?

3 comments
AuthorsOpenReview ↗

Response to Reviewer jjxp (Part 1/3)

We appreciate the reviewer's recognition of our retraction-regret decomposition and broad empirical evaluation. Some concerns appear to stem from PDF rendering issues and misunderstandings of our existing controlled comparisons, which we clarify below. We further address the remaining concerns through additional experiments, a complete theoretical derivation, and discussions of methodological choices and limitations.

Summary of Key Concerns and Main Responses

Key ConcernsMain Responses
Manuscript readability (W1, Q1)Rechecked the PDF and confirmed complete content and correct display in our viewers.
Gain attribution and compute fairness (W2, Q2)Added compute-matched comparisons supporting gains beyond extra compute and dual-endpoint training.
Generated-prefix reliability (W3, Q3)Added stage-wise regret audits and gate-only adaptation experiments on generated prefixes.
Regret decomposition and oracle construction (Q4)Added the full derivation and convexity argument to the main text.
Distribution distillation (Q5)Added justification for distribution matching over direct scalar regression.
Pairing quality and unpaired settings (Q6)Discussed pairing limitations and weakly paired/unpaired extensions as future work.
Training cost and practical value (Q7)Clarified training time and added peak-memory results; gains persist under matched compute.

W1: Please see Q1.


W2: Please see Q2.


W3: Please see Q3.


Q1: Manuscript display and readability.

We thank the reviewer for raising this concern. We have carefully rechecked the manuscript PDF, including Pages 1, 3, and 5, all figures and captions, the methodology, and the appendix, and confirmed that the content is complete and properly displayed in our PDF viewers.

We suspect that the reported issues may stem from a PDF rendering or text-extraction problem rather than missing or corrupted manuscript content. We would greatly appreciate it if the reviewer could kindly reopen or download the PDF and check it again, potentially using a different PDF viewer. We are also happy to clarify any specific pages, figures, or other presentation-related concerns if difficulties persist.


Q2: Compute-matched Comparison.

We thank the reviewer for suggesting this additional analysis. We clarify and strengthen the comparisons in two aspects:

  • Existing comparisons (Sec. 4.3, Table 3). We already compare Standard AR, a dual-endpoint model with a tuned global retraction scalar, and SPIRA. On BraTS T1n→T1c, they achieve PSNRs of 22.16, 22.28, and 22.72, respectively. Since the global scalar and SPIRA share the same dual-endpoint predictor, this comparison demonstrates the benefit of selective token-wise retraction beyond global retraction.

  • Additional compute-matched comparison (Appendix D.7, page 27, Table 20). Following your suggestion, we extend Standard AR training from 100k to 194k steps to match SPIRA's training budget (26.32 GPU hours). Despite the additional training, Standard AR achieves 22.40 PSNR, compared with 22.72 for SPIRA. Under the same budget, Full-prefix AR and the tuned global scalar achieve 22.10 and 22.28 PSNR, respectively.

Table R1. Compute-matched training budget comparison on BraTS 2024 T1n→T1c.

MethodsStepsGPU hPSNR ↑SSIM ↑LPIPS ↓FID ↓
Standard AR (Original)100k13.5722.160.6960.13742.5
Standard AR (Compute-matched)194k≈26.3222.400.6970.13341.4
Full-prefix AR (s=0)100k26.3222.100.6960.13541.9
Full-prefix AR + Tuned global scalar100k25.5722.280.6990.13342.8
SPIRA100k26.3222.720.7050.12638.3

Together, these results show that additional source-only supervision and training compute alone cannot explain SPIRA's gains, while the shared-predictor comparisons isolate the additional benefit of selective token-wise retraction.

AuthorsOpenReview ↗

Response to Reviewer jjxp (Part 2/3)

Q3: Analysis of generated prefixes.

We thank the reviewer for highlighting this training–inference mismatch issue. Following your suggestion, we have added two complementary experiments.

  • Stage-wise local regret on generated-prefix states (Appendix E.2, Table 22, pages 30-31). We divide each 512-block generation window into early, middle, and late stages and evaluate the original, unadapted SPIRA gate against the effect-only gate on identical SPIRA-generated states. Generated prefixes and within-token factor histories are fixed during scoring; GT is used only to measure regret. PRR is reported in %, and magnitude is the mean factor-averaged regret over violating actions, in nats.

Table R2. Stage-wise local regret on generated-prefix states.

TaskGateEarly PRR↓Early magnitude↓Middle PRR↓Middle magnitude↓Late PRR↓Late magnitude↓
BraTS T1n→T1cEffect-only25.70.02027.20.02228.40.024
BraTS T1n→T1cSPIRA14.10.01115.70.01217.90.016
SynthRAD MRI→CTEffect-only28.60.02131.30.02533.10.029
SynthRAD MRI→CTSPIRA15.40.01217.10.01419.70.016
AutoPET CT→PETEffect-only23.80.01526.40.02129.00.024
AutoPET CT→PETSPIRA13.20.00815.00.01017.10.015

As reported, both violation frequency and magnitude increase in later stages, but SPIRA consistently exhibits fewer and smaller violations across all stages and tasks. Hence, its relative advantage persists throughout free-running generation, although the remaining violations confirm that the learned gate does not inherit the oracle’s zero-regret guarantee.

  • Gate-only adaptation to generated-prefix states (Appendix D.6, Table 19, page 27). As suggested, we additionally freeze the tokenizer and AR predictor and adapt only the gate using either teacher-forced states alone or a 1:1 mixture of teacher-forced and generated-prefix states. Both adaptation variants use identical update budgets and the same predictive-action distillation objective. Performance is evaluated through independent free-running generation.

Table R3. Gate-only adaptation to generated-prefix states.

TaskVariantPSNR↑SSIM↑LPIPS↓FID↓
BraTS T1n→T1cOriginal SPIRA22.720.7050.12638.3
BraTS T1n→T1c+ TF-only adaptation22.780.7050.12437.5
BraTS T1n→T1c+ Mixed-state adaptation22.820.7080.12237.2
SynthRAD MRI→CTOriginal SPIRA21.290.7140.22587.3
SynthRAD MRI→CT+ TF-only adaptation21.380.7170.22488.9
SynthRAD MRI→CT+ Mixed-state adaptation21.540.7180.22185.2

As shown, mixed-state adaptation improves all four metrics over both original SPIRA and TF-only adaptation on both tasks, whereas TF-only gains are smaller and less consistent. These results suggest that exposure to generated-prefix states provides benefits beyond additional teacher-forced updates and can further improve free-running generation.

In summary, these experiments support SPIRA’s relative reliability on the evaluated generated-prefix states and show further free-running gains from mixed-state adaptation under matched update budgets.


Q4: Derivation of formulas.

Thank you for this constructive suggestion. Following your advice, we have added the full derivation of the regret decomposition and the convexity argument for the oracle in Sec. 3.3 (Eqs. (5)–(8)) on page 5 of the main text.

Specifically, the added derivation clarifies two key aspects:

  • Regret decomposition (Eqs. (5)–(6)). We explicitly expand the target loss and predictive effect using the softmax normalizing constant. Subtracting these expressions yields the exact decomposition Ri,y(s)=Ei(s)+sAiR_{i,y}(s)=E_i(s)+sA_i, where AiA_i measures the target-relative alignment of the prefix correction.

  • Convexity and oracle construction (Eqs. (7)–(8)). We derive Ri,y′′(s)=Var⁡k∼pi(s)[ri,k]≥0R''_{i,y}(s) = \operatorname{Var}_{k \sim p_i(s)}[r_{i,k}] \geq 0, establishing convexity. Together with Ri,y(0)=0R_{i,y}(0)=0, this guarantees that the zero-regret feasible set forms a closed interval starting at zero, allowing the maximal feasible retraction to be efficiently identified by bisection.

We believe these additions make the theoretical foundation and oracle construction more explicit and easier to follow, directly addressing your concern.

AuthorsOpenReview ↗

Response to Reviewer jjxp (Part 3/3)

Q5: Explanation and justification for distillation.

Thank you for bringing this concern to our attention. Following your advice, we have added an explicit explanation and justification for predictive-distribution distillation in Appendix F (Discussion) on page 32.

Specifically, the gate aims to reproduce the oracle’s predictive behavior rather than its scalar retraction amount s∗s^\ast. Unlike direct regression, KL-based distribution matching accounts for how retraction errors affect predictions, providing a more prediction-aligned supervision signal. This is further supported by the ablation results in Sec. 4.3 (Table 3), where distribution matching consistently outperforms direct s∗s^\ast regression.


Q6: Pairing quality and unpaired settings.

We thank the reviewer for raising this important question.

  • We first clarify that SPIRA does not use flow matching. Instead, the AR predictor is trained with next-block cross-entropy, while the gate is supervised through KL-based predictive-action distillation (Secs. 3.3-3.4).

  • Our experiments follow the paired-data evaluation protocol commonly adopted in recent supervised medical image translation methods [1–5], ensuring consistent training assumptions and fair comparisons. Accordingly, weakly paired or unpaired settings were not included in our current evaluation.

  • We acknowledge that imperfect pairing may affect target-loss evaluation, oracle supervision, and gate reliability. While this sensitivity has not yet been quantified, we have added a discussion in Appendix F (Discussion) on page 32 and identified extending SPIRA through registration-aware supervision, pseudo-pairing, or consistency-based learning as promising future work.


Q7: Training Cost.

Thank you for raising this practical concern. We clarify the training costs and provide additional evidence in two aspects:

  • Training and inference costs (Appendix E.3, Tables 23–24, pages 31–32). We have reported the total training time and inference efficiency, and now additionally include peak training GPU memory as suggested. Compared with Standard AR, SPIRA requires more training time (26.32 vs. 13.57 GPU hours, 1.94×) and peak training VRAM (21.86 vs. 12.14 GB, 1.80×). Its reported peak training memory matches the other dual-endpoint variants. At inference, SPIRA adds modest latency over Full-prefix AR without increasing peak VRAM.

  • Additional compute-matched comparison (Appendix D.7, Table 20, page 27). To further assess the quality-compute trade-off, we extend Standard AR training to 194k steps to match SPIRA's training budget (26.32 GPU hours). As reported, under identical compute budgets, SPIRA still outperforms Standard AR, with consistent improvements across all four metrics.

Together, these results show that SPIRA improves translation quality under matched training compute, while the learned gate adds little overhead beyond dual-endpoint prediction.


We thank the reviewer once again for the insightful comment. We believe these clarifications, additional experiments, and theoretical details help resolve the reviewer's concerns. We would be pleased to provide further clarification or engage in additional discussion if needed.


References

[1] Meng X, et al. Multi-modal modality-masked diffusion network for brain mri synthesis with random modality missing. IEEE TMI, 2024.

[2] Ha J, et al. Multi-resolution guided 3d gans for medical image translation. WACV, 2025.

[3] Arslan F, et al. Self-consistent recursive diffusion bridge for medical image translation. MedIA, 2025.

[4] Pang H, et al. Cascaded diffusion model and segment anything model for medical image synthesis. PR, 2026.

[5] Dorent R, et al. Unified cross-modal medical image synthesis with hierarchical mixture of product-of-experts. IEEE TPAMI, 2026.

Reviewer_4286OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SPIRA translates 3D medical images by balancing information from the source image and previously generated blocks. During training, the known target guides a gate that learns when to reduce the influence of earlier blocks. Experiments on eight tasks across BraTS 2024, SynthRAD 2025, and AutoPET show improvements over the tested baselines. The idea is interesting, and the evaluation includes useful controls. However, the gains are modest on most tasks, and important comparisons are still missing. I would need stronger evidence of the method’s added value to support acceptance.

Critical strengths
  1. Earlier generated content can help prediction but can also introduce errors. The source-only and full-prefix comparisons use the same trained model, which helps show the benefit of controlling this influence.

  2. The evaluation covers eight translation tasks involving MRI, CT, and PET. Results from three training seeds on BraTS, lesion-level tests, and tumor segmentation provide additional support for the reported gains.

Critical weaknesses
  1. SPIRA’s gains over the best external baseline are modest on most tasks in Tables 1 and 2. On seven of the eight tasks, PSNR improves by 0.35–0.78 dB and SSIM by 0.006–0.011. The paper does not explain whether these small gains reflect the difficulty of the task or limits of the proposed method.

  2. The paper discusses NWCAD and VPG, but does not test adapted versions of them. The internal gate comparisons are useful, but do not establish an advantage over these existing methods. A comparison using the same model and data would make SPIRA’s added value clearer.

Questions for authors
  1. What limits the improvements on most tasks in Tables 1 and 2: missing information in the source images, details lost during tokenization, or limits of the gate?
  2. Can the authors compare SPIRA with adapted versions of NWCAD or VPG using the same model and data?
  3. How does SPIRA compare with Standard AR when both receive the same training budget?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 4286 (Part 1/2)

We sincerely thank the reviewer for recognizing our controlled source-prefix comparisons and broad empirical evaluation. In response, we clarify the value and limitations of the modest gains, add comparisons with adapted NWCAD–CoCoA and VPG, and provide a compute-matched evaluation to further assess the benefit of selective retraction.

Summary of Key Concerns and Main Responses

Key ConcernsMain Responses
Modest gains and potential bottlenecks (W1, Q1)Clarified consistent gains, downstream benefits, and potential bottlenecks.
Comparison with related methods (W2, Q2)Added matched comparisons with adapted NWCAD–CoCoA and VPG; SPIRA outperforms both.
Training-budget fairness (Q3)SPIRA retains its advantage over Standard AR under matched training compute.

W1: Please see Q1.


W2: Please see Q2.


Q1: Modest Improvement.

Thank you for raising this important concern. We agree that the gains over the strongest external baselines are modest on most tasks, and that their scope and limitations deserve a clearer explanation. We address the value of these gains and the factors that may constrain them below.

  • Consistent improvements in translation quality and benefits for downstream applicability. Although modest, the improvements are consistent across all eight translation tasks and all four reported metrics in Tables 1–2, covering both reconstruction accuracy and perceptual quality. The benefits also extend to downstream tumor segmentation: Appendix D.4, Table 14 reports the best average Dice compared with the strongest external baseline, with SPIRA leading on three of four translation directions.

  • Quantitative and qualitative analyses support the proposed mechanism. Secs. 4.4–4.5, Figs. 5–7 explain why similarly sized prefix effects can have different target utilities and how selective retraction improves generation. The quantitative analyses link better retraction decisions to generation gains, while the qualitative examples illustrate both the removal of harmful corrections and the retention of useful prefix information. These complementary results support the effectiveness of our proposed mechanism beyond the aggregate performance differences.

  • What limits the improvements? Our current experiments do not establish a single dominant bottleneck. Both task-related uncertainty and limitations of the proposed method may contribute:

    • Source information. A single source modality may not uniquely determine target-specific details. Adjusting prefix reliance cannot eliminate this underlying uncertainty or guarantee recovery of subject-specific information absent from the available evidence.

    • Tokenization. Lossy compression may discard fine details and restrict reconstruction fidelity. Nevertheless, our tokenizer reconstruction results (Appendix B.1, Table 6, page 16) do not establish tokenization as the dominant explanation for the modest gains.

    • Gate accuracy and available retraction opportunities. SPIRA adjusts predictions between the source-only and full-prefix endpoints, which limits its correction capacity. Fig. 6(c) shows larger gains when more prefix correction can safely be retracted, while Fig. 6(a) indicates that the learned gate still has room for improvement relative to the oracle.

    • Optimization objective. The oracle is defined using token-level target loss, which need not align perfectly with decoded-image quality or downstream utility. Directly incorporating these objectives is a potential direction for further improvement.

We have added this discussion to Appendix F (page 32), distinguishing the limitations supported by our analyses from possible contributing factors that require further investigation. Taken together, the consistent translation gains, improved average downstream performance, and mechanism analyses support the value of SPIRA despite its modest absolute improvements. We appreciate the reviewer’s feedback and welcome further discussion and suggestions.

AuthorsOpenReview ↗

Response to Reviewer 4286 (Part 2/2)

Q2: Additional external baselines.
We thank the reviewer for this valuable suggestion. NWCAD [1] and VPG [2] were originally developed for context-conditioned and visual autoregressive generation rather than 3D medical image translation. Thus, our initial evaluation focused on translation-specific methods and controlled internal baselines. To address this concern, we have added comparisons with adapted NWCAD–CoCoA and VPG in Appendix D.6 (page 26, Table 18).

For a fair comparison, both methods are adapted to our 3D blockwise AR framework using the same frozen dual-endpoint predictor, tokenizer, datasets, and evaluation protocol as SPIRA. Specifically, NWCAD–CoCoA operates on source-only and full-prefix predictions, while VPG contrasts clean-prefix predictions with corrupted-prefix predictions using same-block full-embedding replacement. Both retain their original hyperparameter settings and perform independent free-running generation.

Table R1. Comparison with adapted context/prefix decoding methods.

MethodsBraTS PSNR↑SSIM↑LPIPS↓FID↓SynthRAD PSNR↑SSIM↑LPIPS↓FID↓
NWCAD–CoCoA21.700.6920.15747.020.670.6980.24598.5
VPG21.980.6960.14245.220.810.7000.23796.4
SPIRA22.720.7050.12638.321.290.7140.22587.3

As shown above, SPIRA outperforms both adapted baselines across all four metrics on both tasks. These results further support the benefit of learned, region-wise prefix retraction over existing context/prefix decoding strategies under the same predictor and evaluation setting.


Q3: Compute-matched Comparison.
Thank you for your insightful suggestion. Following your advice, we have added a compute-matched comparison in Appendix D.7 (page 27, Table 20). Specifically, we extend Standard AR training from 100k to 194k steps to match SPIRA's training budget (26.32 GPU hours). We also include Full-prefix AR and a tuned global retraction scalar using the same dual-endpoint predictor for a controlled comparison.

Table R2. Compute-matched training budget comparison on BraTS 2024 T1n→T1c.

MethodsStepsGPU hPSNR ↑SSIM ↑LPIPS ↓FID ↓
Standard AR (Original)100k13.5722.160.6960.13742.5
Standard AR (Compute-matched)194k≈26.3222.400.6970.13341.4
Full-prefix AR (s=0)100k26.3222.100.6960.13541.9
Full-prefix AR + Tuned global scalar100k25.5722.280.6990.13342.8
SPIRA100k26.3222.720.7050.12638.3

As shown, although extending Standard AR training improves its performance, SPIRA still achieves better results under the same compute budget. Moreover, SPIRA outperforms both Full-prefix AR and the tuned global scalar using the same predictor, confirming that its gains are not merely due to additional training compute or source-only supervision, but also benefit from selective token-wise retraction.


We sincerely thank the reviewer again for the constructive feedback. We trust that these clarifications and additional experiments provide sufficient evidence to alleviate the reviewer's concerns. We remain open to further suggestions and would be happy to discuss any remaining questions.


References

[1] Tao Y, Agrawal A. No-worse context-aware decoding: Preventing neutral regression in context-conditioned generation. ACL 2026.

[2] Liao X, et al. VPG: Visual Prefix Guidance for Autoregressive Image and Video Generation. arXiv preprint arXiv:2605.30317, 2026.

Thank you for the clear response and the new experiments. The comparisons with adapted NWCAD–CoCoA and VPG and the compute-matched comparison answer my main questions. I also appreciate the honest discussion of what may limit the gains. However, the gains remain modest, and the new comparisons cover only one or two of the eight tasks. The adapted baselines also keep their original settings instead of being tuned for this task, so the gap might be smaller after tuning. I will keep my score of 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

I. Acknowledgments

We thank the AC and all three reviewers for their careful evaluation and constructive feedback. Their comments have helped us strengthen the theoretical explanation, empirical evidence, and discussion of limitations. Below, we summarize the recognized strengths, our main responses, and the completed revisions.


II. Key Strengths

The reviews highlighted the following strengths:

  • Conceptual Insight: Distinguishes predictive effect from target utility, emphasizing the role of target alignment (jjxp, KM3r).

  • Principled Formulation: Provides a clear retraction–regret decomposition for guiding prefix retention (jjxp).

  • Broad Evaluation: Extensive 3D medical translation experiments with controlled comparisons and informative ablations (4286, jjxp).


III. Key Concerns and Main Responses

Key ConcernsReviewersMain Responses
Modest gains and practical value4286Clarified consistent improvements, downstream benefits, and potential limitations.
Related decoding baselines4286Added adapted NWCAD–CoCoA and VPG comparisons. SPIRA outperforms both.
Compute fairness and training cost4286, jjxpAdded compute-matched comparisons and peak training memory. SPIRA retains its advantage.
Generated-prefix reliabilityjjxpAdded stage-wise regret analyses and gate adaptation experiments, confirming improved reliability.
Theoretical derivationjjxpAdded the full regret decomposition and oracle convexity derivation.
Gate design and distillationKM3r, jjxpClarified the MLP design, input features, and distribution distillation using existing ablations.

IV. Additional Clarifications

We also clarify several concerns related to the existing manuscript:

  • PDF completeness (jjxp): We rechecked the manuscript and confirmed that its content, figures, and methodology are complete and properly displayed in our PDF viewers.

  • Existing controlled comparisons (jjxp): Sec. 4.3, Table 3 already compares Standard AR, tuned global retraction, and SPIRA. The new compute-matched experiment further strengthens this evidence.

  • Target alignment and learned gate (KM3r): Target alignment determines the training oracle, while the learned gate approximates its decisions using GT-free features, as supported by Figs. 5, 6(b), and 10(a).

  • Scientific presentation (KM3r): Question-style subheadings and highlighted boxes are intended to improve readability while maintaining scientific rigor.


V. Revisions and Updates

The main additions are highlighted in purple\color{#5941B5}{\text{purple}} in the revised manuscript:

  • Theoretical derivation — Sec. 3.3: Added the complete retraction–regret derivation and the convexity argument supporting oracle construction.
  • Additional comparisons — Appendices D.6–D.7, Tables 18 and 20: Added adapted NWCAD–CoCoA/VPG baselines and the compute-matched Standard AR comparison.
  • Generated-prefix analyses — Appendices D.6 and E.2, Tables 19 and 22: Added gate-only adaptation and regret audits across early, middle, and late generation stages.
  • Cost reporting — Appendix E.3, Table 23: Added peak training GPU memory alongside the existing computational cost measurements.
  • Expanded discussion — Appendix F: Added explanations of distribution distillation, modest gains and potential bottlenecks, and paired-data limitations.

We thank the AC and reviewers again for their time and valuable feedback. We hope these clarifications and additions provide a clearer basis for assessing our paper, and we welcome further questions and discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.