- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies fraud inspired by many-armed bandit setting, where adversarial arms can also manipulate endorsements. It gives a lower bound linking fraud to the number of distinct arms tried when side information cannot identify attackers. It also studies how interaction patterns can help detect groups of adversarial arms and tests several defenses against manipulated social evidence.
Critical strengths
-
The paper studies an problem: social evidence can help learners find good arms, but adversarial arms can also create misleading evidence.
-
Theorem 1 and 2 offer a useful contrast between cases where side information reveals nothing about adversarial arms and cases where group structure supports detection.
Critical weaknesses
Weakness: The methods and experiments are hard to follow, making the main contributions difficult to assess.
The paper provides individual definitions and defensive rules, but readers must reconstruct how they fit together across several sections and the appendix. This makes it difficult to understand what each method does, which guarantees apply to it, and what the experiments establish. The following examples illustrate this broader problem.
Section 4 would benefit from end-to-end pseudocode showing how arm selection, rating updates, filtering, and blocking work together. Explicit ETS/ETS-sep sampling equations and a table of method configurations would also help. If graph components change online, the handling of their CUSUM statistics needs clarification.
Section 5.1 summarizes a large experimental suite, but provides no table or figure showing results by condition or the size of the differences between methods. Readers therefore cannot directly assess the claims of “least fraud” and “no loss in reward.”
Table 2 reports regret without attackers and fraud under attacks. These results show separate benefits, but do not show the reward cost of the defenses under attack. The paper should clarify whether “the regret gain intact” refers only to performance without attackers, or provide reward results under the same attack conditions.
The paper does not need to provide a single best algorithm or a joint regret–fraud bound. However, it should clearly explain what its cost analysis establishes, which design principles follow, and how the proposed methods and experiments support those principles.
Questions for authors
-
Could you provide concise end-to-end pseudocode and explicit ETS/ETS-sep sampling equations, specifying the timing of rating updates, filtering, graph construction, and blocking?
-
Section 4.1 states that a 30-member ring produces “30 times as many observations per round,” although Section 2 allows only one arm pull per round. Could you explain this statement?
-
I find the motivation of studying collusion and defenses in many-armed bandits interesting. I would like to understand the choice to focus on fraud rather than also analyze regret. Were there particular technical difficulties in connecting the two? If so, could you explain the main obstacles?