- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposed a method to integrate camera embeddings for the TTT layers. The paper carefully analyzed where and what kind of camera embedding works the best, as well as the analysis extends to a wide range of fast weight networks. The paper shows
Critical strengths
- The design itself is quite interesting. It integrates the camera embedding to the TTT layer which is novel and it makes sense. The proposed input, hidden, value/output embeddings with norm-preserving transformations and extension across different fast-weight networks makes it a pretty principled architecture design.
- The experimental results is overall solid. The comparison shows the improvements across three datasets. Multiple different fast-weight networks are tested as well.
Critical weaknesses
overall i think it is a pretty clear paper with sound ablation results, so i do not have many concerns. my main concern is on the results reported in table 3. On the baseline part, i think the absolute NVS quality is also kind of on the lower side if compared to recent papers like RayRoPE or URoPE. i understand that people establish their own baseline and do fair comparison on it, but still it raises some questions about whether the gains persist with stronger baselines. On the RayRoPE baseline: it is adapted from query-centric to world-frame coordinates, and GTA's output transformations are not clearly specified. I think more implementation details and analysis would make the results more solid.
Questions for authors
N/A
I'd like to thank the authors for their detailed response.
My concerns about softmax-attention baselines were addressed.
My second concern about new geometric constraints was partially addressed. Applying the encoding on the hidden state is technically sound, but in my opinion an architectural requirement from TTT. The authors confirmed in their response that there is just no equivalent in softmax attention. The idea and justification of orthogonal matrices is valuable and a good insight. Furthermore, my initial comment also stated that this work "does not add additional geometric constraints." The authors did not comment on this in their response. Similar to RayRoPE, they keep ray+point embeddings, but remove invariance to the world frame. I don't see what new geometric (or projective) constraints are modelled.
The analysis on world-frame perturbation is interesting.