- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper is clearly AI written and not very understandable. My best understanding is that the authors [or their AI] are investigating how LLM skills can be audited for malicious content and to what extent ensembling helps and how it depends on ensemble members being from same / different families. As one would expect, different families help all else equal, but because in real world all else is generally not equal, they find that it does not actually always help. The paper is trivial and lacks any non-trivial scientific insights imo and should be rejected.
Critical strengths
No strengths. This is an AI slop paper that should be strongly rejected.
Critical weaknesses
The paper is AI written and incomprehensible to me as a human reader
The paper is not readable and seems like AI generated. I am highlighting specific issues just within first section of section 2.2 PROBLEM FORMULATION (pasted below):
A promotion service admits reusable skills to an agent library using observer reports about fixed artifacts. Later execution is evaluated separately. Let C be a candidate skill and Y ∈ {0, 1} its evaluation label, with Y = 1 denoting malicious content that should not be promoted. Observer i receives a registered view Xi(C) and reports clear, protected, or insufficient. Let Vi = 1 denote passing schema, cardinality, identifier, and binding checks. Complete-clear requires Vi = 1, complete coverage, untruncated input, and a clear verdict. The formal fail-closed map sets Ai(C) = 1 for complete-clear and 0 otherwise.
- What is a promotion service?
- What is an observer report?
- "Later execution is evaluated separately" -> air-dropped sentence with no connection to surrounding context and unclear meaning. No explanation of why is this evaluated separately..
- Bunch of symbols are dropped without any clear explanation of their context and how they are related.
- lots of nouns with ambiguous meaning; cardinality, what is it referring to here? what is an identifier? what is binding check?
This is not at all comprehensible to me (a human). I have read the problem statement multiple times and can not say I understand it in a satisfactory way. And to stress this issue is widespread through the whole paper.
There is no clear research contribution made by the paper
The paper has no clear contribution / findings in my view. The paper is, as most AI slop papers do, projecting some run-of-the-mill results as non trivial (e.g., propositions 1 and 2 in the paper as far as I can tell follow straightforwardly from the modelling assumptions, which seem to be non-trivial and insufficiently justified, and also these propositions do not contribute anything).
The figures are AI generated and AI usage statement is incorrectly claiming otherwise
AI usage statement says AI was not used in preparation of figures, but that is clearly false. My sense is that even experimental design etc. was also done by AI and AI usage statement is clearly misrepresenting / underreporting AI use.
Questions for authors
N/A
Dear Reviewer,
As the deadline for the discussion period approaches, we kindly request your feedback on our response. Your insights would be greatly appreciated.
Thank you for your time and consideration!