- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
This paper claims to investigate cases where an LLM can correctly apply individual constraints to the problem at hand, but fails to generate a solution that satisfies their composition/conjunction. The paper is so badly written that it is not clear that anyone could follow it, perhaps even the authors.
The target audience for this work is so narrow as to make this paper completely unnacceptable, it also means its hard to take away any general results from the work. This work appears to be the result of GenerativeAI which generates such a jargon filled mess that leaves it beyond human understanding.
Critical strengths
None
Critical weaknesses
The writing is in general incomprehensible.
The abstract is not standalone and essentially incomprehensible.
l.20. L2/L4 are meaningless here, and not ever really defined in the paper.
l.51. "enter the residual denominator" how are readers meant to understand this phrase.
l.53. "L2-to-L4 increase": meaningless
l.175. You define F(X,C) is you want to fix an X and C and use the notation F to represent F(X,C) you need to declare this.
l.176. The notation I(C) is not defined.
l.188. The "matched ring gate" is not defined here, and even later not properly defined.
Fig 1. What are "frozen strict denominator" and "stateless joint construction" they need discussion and explanation
l.210. Too terse to be comprehensible, e.g. "G_{pair} matched coordinator interfaces and non-joint checks"
l.218. Notation Y_{1:t} is not defined
l.233. The Matched Constraint Ring is not defined with enough detail to understand. Are we mean to understand that composing O_L ... O_2.O_1 gives the identity function, or that the composition has exactly one of the 5 symbols that for X_1 will map back to the same X_1 value.
l.239. "retains inventories" what is this meaning, we choose O_1, .., O_L so that there is exactly one fixpoint of the composition, i.e. one value of X_1 which is a fixpoint. Make this clear.
l.239. The notation L2 and L4 are not defined. Here we can guess you mean L = 2 and L = 4 in the definition above.
l.249. "One source is pinned" explain what this means.
l.256. What does it mean to freeze 50 paired seeds?
l.264. "Ring length is a dose variable" incomprehensible
l.274. "Updo" is never defined or given a reference
l.275. "Pebble" is never defined or given a reference
l.453 "T = 0" T is never defined or explained
Questions for authors
Can you define the research question you tackle clearly, and explain the results in terms of what we learn from them!
Post rebuttal:
The authors attempted to answer my questions clearly, this should have been done in the original submission. While the rebuttal does clear up some of the ambiguity, the underlying weakness of the experiment and lack of clarity still means this has to be rejected.