- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DimPO is dimensionality reduction technique that takes inspiration from listwise preference optimization for reducing dimension of query and key spaces. The DimPO objective, which is trained offline, contains the PO term and a top-K cross-entropy term that optimizes for keeping the new probability aligned to the original while pushing the mass to top-K key positions. The evaluations are broad as it contains multiple benchmarks, context lengths, compatibility with different KV reduction methods, and ablations over the layers affected and top-K selection.
Critical strengths
- The objective of DimPO is well-motivated.
- Experiments support DimPO’s benefits against its peers on varying context-length along with its compatibility with SnapKV and MagicPig.
- Ablations over reduced dimension size, layers affected, and top-K selection is good.
Critical weaknesses
- The notations in section 3.2 are hard to follow as y_i and x are not clearly defined. I understand they are adopted from PO’s literature but be explicit here as the application is non-trivial.
- Positional encoding is not discussed enough. If RoPE is used then it will depend on the dimension of query and key space so how does DimPO work there? Should the projection be applied before or after applying RoPE? I am assuming that for generalizability of the method to arbitrary PE, DimPO may be applied to raw queries and keys but would like to be clarified.
- For causal language modeling, are the ranking lists generated using the past keys only? The text is not clear about it.
- While the evaluations are broad along with ablations, the benefit of DimPO is not yet convincing and well-tested as the accuracy drops with KV memory reduction follows the same trend as its peers without a clear sign of superiority in Fig 5-9. The plot may be misleading me for the worse so please provide numbers for sanity check.
- The latency chart in Fig 11 is missing the peers. It would be good to see the accuracy-latency tradeoff, if any, of DimPO against its peers.
Questions for authors
No specific question apart from the concerns mentioned in the weaknesses.
We thank the reviewer for the positive evaluation and constructive feedback. We appreciate the careful notes on missing technical specifications, notation details, and parts of the text that needed clearer explanations. We have addressed these points directly in the revised manuscript and reuploaded the PDF. Below we also provide detailed point-by-point responses.
1. Notation in Section 3.2
In Section 3.2, x represents the query vector at a given position, yi denotes the i-th key vector, and ψi is the original attention weight assigned to that key by the frozen model. The preference loss ranks candidate keys relative to the query using projected Q⋅K scores (si=βlogπθ(yi∣x)). We have added this explicit notation bridge in the revised section.
2. Positional Encoding (RoPE)
We apply full-dimensional RoPE to Q and K first, and only then apply the projection F to reach dimension d′. RoPE is never applied in the reduced space. Because the backbone computes attention scores from post-RoPE representations, DimPO operates in the exact same post-RoPE geometry. Projecting before RoPE would mix frequency pairs and distort positional relationships. This execution order is now stated in Section 3.1.
3. Causal LM and Ranking Lists
Queries at any position t are ranked against the entire 4096-token key set, including positions after t. Because F is a single position-independent map, it is trained to preserve query--key ranking and preference geometry across full contexts rather than fragmented short prefixes. The causal mask is applied subsequently at inference, identically for both original and projected models. We have explicitly added this clarification to the beginning of Section 3.3 in the revised manuscript.
4. Performance Comparison Across Methods
Because Figures 5–9 in Appendix F split individual harness tasks into narrow subplots, visual scale can obscure absolute gaps. However, Figure 2 in the main body presents the exact 5-task average for LLaMA 8B, showing that DimPO consistently outperforms baseline objectives, particularly pairwise variants.
We have expanded Appendix F in the revised PDF with complete per-task, per-model, and per-layer numerical tables. Across these full metrics, DimPO trails slightly only on TruthfulQA (LLaMA 3B/8B) and occasionally on isolated layers for ARC-Challenge (LLaMA 3B, Qwen 7B) and WinoGrande (LLaMA 8B, Qwen 7B). However, on average, DimPO still achieves the highest overall scores.
Below are the 5-task averages for LLaMA-3.1-8B and Qwen3-4B, alongside ARC-Challenge for LLaMA as a representative task breakdown. Full numerical evaluations for all models and tasks are provided in Appendix F.
LLaMA-3.1-8B — 5-task average (d′=64)
LLaMA-3.1-8B — ARC-Challenge only (d′=64)
Qwen3-4B — 5-task average (d′=64)
5. Latency Trade-off (Figure 11)
All objectives share the exact same architecture, linear projection path, and inference graph. Consequently, runtime latency is identical across methods for a given layer depth l. The performance difference lies entirely in task accuracy at that shared speedup, which is why Figure 11 visualises DimPO's latency only.
To be completely transparent, below is the full accuracy-latency comparison at layer depth l=16 (~12.5% KV cache memory reduction for LLaMA-3.1-8B). Latency is the mean of 8 \texttt{generate} calls after 1 warmup call, each with batch size 4, an 8192-token prompt and 2048 generated tokens, on a single NVIDIA RTX PRO 6000:
All methods deliver the same ~8.3% speedup at l=16, but DimPO is the only preference-based objective that retains long-context performance.