- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates an interesting problem: a model can be nearly insensitive to individual identity substitutions while still depending on the correspondence among multiple inputs. To study this phenomenon, the authors introduce behavioral quotients to formalize identity abstraction and quantify the information lost when multiple identity distinctions are removed jointly. Through theoretical analysis, the authors establish bounds connecting local substitution discrepancies to global quotient errors and demonstrate a complexity separation between independent block quotients and binding-preserving coordinated quotients. The findings are further supported by controlled Transformer experiments. And the paper provides useful insights into how relational dependencies can be distributed across many weak local effects, highlighting the importance of evaluating joint abstractions rather than relying solely on individual substitutions.
Critical strengths
-
This paper is well-written and clear. All the mathematical derivations are rigorous. It establishes explicit bounds between local substitution discrepancies and global quotient errors, provides finite-sample certification guarantees, and carefully derives the main theoretical results.
-
The paper investigates an interesting and meaningful research problem. It highlights an important distinction between local identity substitutability and joint abstraction, showing that small changes under individual substitutions do not necessarily imply that multiple identity distinctions can be safely removed together. This provides a useful perspective for understanding relational dependencies and entity binding.
Critical weaknesses
- Although Theorem C establishes a rigorous and interesting complexity separation between independent block quotients and binding-preserving coordinated quotients, its broader implications for relational abstraction in practical models could be further discussed.
Questions for authors
-
There are several cross-reference formatting issues throughout the manuscript. For example, Line 178 has a typo: "Equation equation 9," and more cases occur in the Appendix. Please modify it.
-
I have mentioned this as Weakness 1. I appreciate that the authors explicitly define the scope of their theoretical results and acknowledge the challenges of extending them to pretrained LLMs. However, a more concrete discussion of how these quotient families and transformation assumptions might correspond to entity binding or relational reasoning in LLMs would strengthen the practical relevance of the theoretical findings. While additional formal results or LLM experiments may be beyond the scope of this paper, clarifying these connections would provide useful guidance for future research.
-
I think the theoretical novelty of Theorem A could be better distinguished from existing work. For example, Filmus [1] has already studied functions over fixed-Hamming-weight slices of the Boolean hypercube and their associated spectral properties, which are closely related to the derivation of Theorem A. Although the authors cite this work in the Related Work section, I suggest providing a more explicit discussion and relevant citations alongside the theorem and its derivation to better clarify which results follow from existing theory and what constitutes the new contribution of this paper.
Reference
[1] Filmus, Y. (2016). An orthogonal basis for functions over a slice of the Boolean hypercube. The Electronic Journal of Combinatorics, 23(1), P1.23.
3 comments
Thank you for response
Dear authors,
Thank you so much for the thoughtful response. I think extending this work to large language models as future work will be interesting. I will maintain my score.
One small suggestion: in your revised manuscript, please try to use a different color to highlight the additions or modifications made during the rebuttal phase, so that it is more convenient for our discussion.
Thank you.
Dear Reviewer NUM5,
Thank you again for your suggestion and positive assessment of our work. We appreciate your recognition of the paper's contributions.
Given your overall positive evaluation and the clarifications provided during the discussion, we would greatly appreciate it if you could reconsider whether the paper now merits a Clear Accept (4). If you still have any concerns preventing a higher rating, we would be happy to address them.
Thank you very much for your time and consideration.
Dear Reviewer NUM5,
We would like to respectfully offer a reminder and invite you to join the further discussion. We believe that most concerns have been reasonably addressed or substantially resolved. If you have a different view, we can give further clarification. Alternatively, we would also welcome you to raise the score or adjust the confidence.
We would also like to share important updates that may change your view of the paper. Following your suggestions, we have strengthened the discussion of practical implications and clarified our theoretical contributions.
We have also corrected the cross-reference issues and expanded the discussion of related work.
We look forward to your reply and will be happy to answer any further questions.