- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Submission research on the representation usage gap of embedded spatial diffusion language models under severe code compression. In the TinyStories k=16 setting, direct conditioning will still lose control of one to three reliable attributes even if the probe can still recover attribute information from compact code. Residual prediction (difference between clean embedding and projection code) and additive reconstruction after sampling can restore all six attributes and improve control on BeerAdvocate. Interventions such as input summation and continuous residual intensity scanning are used to restore control in a fixed code dimension, pointing to weak code denoising coupling. Both neutral code guidance and Feynman Kac particle steering can improve both types of models, but the residual host maintains an advantage. The same residual intervention cannot improve the ELF flow model and establish generator family boundaries.
Critical strengths
The experimental separation design between capacity and generation usage is detailed. The probe capacity ratio directly compared to the residual arm is almost identical to the logical readout, while the output control is clearly differentiated, cleanly demonstrating the phenomenon of information that may exist but is not effectively utilized. Residual training has been proven to complement the inference controller: under the guidance of matched neutral code and FK steering, the residual direct gap still remains, and the control comparison further reveals the different trade-offs between leakage diversity computation.
Critical weaknesses
W1. Residual intervention simultaneously changed the prediction target, added noise variables, and reconstruction path. Although the author attempted to isolate the coupling strength through continuous g-scan and additive path ablation, residual transformation still altered the diffusion process itself. It is still unclear whether the restoration control stems from stronger code usage or fundamentally different noise to signal mapping.
W2. The number of seeds is extremely small. Most comparisons rely only on two to three seeds, and several key tables only report the mean of seed-1 or seed-1/seed-2. Such limited repetition results in a lack of statistical robustness in the reported axis loss patterns and parity differences.
W3. The residual advantage could not be transferred to the ELF conditional flow, although it represents pressure matching. The boundary of the generator family is recognized but no mechanism explanation is provided. If it is not clear why residual transformation helps DDPM denoising but is ineffective for popularity, the practical scope of the proposed remedial measures is still narrow.
W4. Evaluate the dependency on an arbitrary 0.8 odd even threshold and a carefully selected subset of six attributes. The length is uncontrollable in all compact arms, and the vehicle is marginalized; Their exclusion has shaped the headline proposition of "all six reservations". The sensitivity table exists, but the main narrative is still based on a single threshold and attribute set.
W5. Both corpora are relatively small and have limited styles. TinyStories is a short composite narrative, while BeerAdvocate is an aspect rating review. The lack of experiments with larger and more diverse text distributions or training of unfrozen T5 embeddings from scratch limits the claim of insufficient use of general conditioning.
W6. The generation of quality indicators is in a secondary position. Once the control is equalized through guidance, residual models often exhibit higher perplexity or lower coherence. The paper correctly points out the trade-off, but does not quantify whether the control gain of the residual host is sufficient to compensate for the quality cost in real downstream use.
W7. Numerical patterns appear in the table: odd and even values often cluster at. 9x or 1.xx, axis loss counts are clean integers across seeds, and several capacity ratios fall within neat intervals of. 375/. 400/. 425. Although it cannot prove fraud, these patterns deserve a more careful examination of the evaluation assembly line and the original seed level trajectory.
W8. There is almost no theoretical or analytical support. Work is purely experiential; Why does the residual target improve the gradient flow through the prefix path, or how does additive reconstruction interact with the attention of the denoising unit on the code token.
Questions for authors
If both arms are forced to share identical noisy variables and reconstruction paths, and only isolate the predicted target, can the residual direct difference still be reproduced?
What is the mechanism difference between DDPM denoising and ELF flow that leads to complete failure of residuals in the latter?
If the token is embedded as learned rather than frozen from T5 small, will the same representation use gap and residual repair still occur?