- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents an empirical audit of learned message selection under communication budgets. By evaluating all 32 subsets of five candidate messages per question, it distinguishes whether a successful subset exists, whether a selector identifies it, and whether its success persists across executions. It also examines whether optimal selections remain compatible as the budget increases.
Critical strengths
None
Critical weaknesses
The fixed pool of five messages lacks sufficient justification or an ablation over pool size. With so few candidates, restrictive budgets, such as forwarding only two messages, require a clearer practical motivation. Larger pools would make this motivation more compelling, but the exhaustive audit requires 2^M receiver evaluations for M messages. The tension between a practically motivated selection problem and computational tractability remains unresolved.
The paper measures context-dependent message effects but does not systematically model or control whether the underlying evidence is complementary, contradictory, or redundant. These relationships are central to the selection problem. Training selectors on subset outcomes without examining how these relationships affect their behavior provides limited explanation of what makes selection difficult or why the learned policies fail.
The contribution is primarily an evaluation framework rather than a new selection algorithm. Its value therefore depends on the insights gained from the audit. However, the findings remain largely descriptive, and the paper does not clearly establish what new, generalizable guidance they offer for designing effective selectors.
Questions for authors
What practical constraint motivates the five-message setting and its budgets? Can the authors vary the candidate-pool size and report how the main findings and computational costs change?