- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes B4I, a two-stage framework for predicting item popularity without platform internals or feedback on the target item.
- Stage I learns a neural-logical surrogate from historical user-group–item interactions.
- Stage II applies it to a fixed user-group panel and aggregates the resulting scores and rule representations to classify each item into one of four ordered popularity levels.
Experiments compare B4I with direct prediction baselines, along with ablations and a qualitative analysis of the learned rules.
Critical strengths
S1. Creators often need to estimate how content will perform and how audiences may differ before any feedback is available. The paper addresses this need directly.
S2. Using historical user–item data to inform item-level prediction is well motivated compared with training an item classifier alone. The fixed user panel also gives a clear and consistent basis for comparison.
Critical weaknesses
W1. Appendix D.2 cites FENCR, which already learns neural-logical rules over user/item feature embeddings, uses data-driven rule selection, and produces rule-level explanations with learned aggregation. So the capability itself is not new.
Is Stage I built on FENCR? If so, what was changed (e.g., ID pathway, rule operators, training objective, contribution computation)? If not, please give a full specification of , , and RuleEnc rather than the current abstract description.
W2. , , RuleEnc, and the Stage II modules (, , , ) appear only as abstract mappings. It is unclear how rules select features and form conditions, how logical operators are implemented, how rule number and length are set, what RuleEnc actually encodes, and which pooling is used in the experiments. Dimensions and training configuration are also missing.
Since the neural-logical surrogate is a claimed contribution, this goes beyond missing hyperparameters: the method cannot be reproduced, and its difference from FENCR cannot be assessed.
W3. Reported MAE values are inconsistent with the stated definition (§3.4, Table 2). §3.4 defines MAE as the error between predicted and true class indices. If predictions are obtained by argmax, then with 72 test items and 3 seeds, the mean MAE must be an integer multiple of . The values in Table 2 for LR (0.7484) and B4I (0.5814) do not satisfy this constraint, and four-decimal rounding cannot account for the gap (e.g., , ; , ).
Please clarify whether MAE is computed from the expected class, uses sample weights, or is aggregated differently, and state whether MSE and MAE use the same prediction transformation. Releasing the evaluation code would help resolve this.
W4. The appendix claims that requiring ≥10 impressions with zero clicks avoids mislabeling under-exposed items. But most iPinYou campaigns have CTR below 0.1%, and even at a true CTR of 0.1%, . I note that the selected items are those with the most impressions, so the actual exposure may be much higher. Still, please report the impression distribution of the 177 items to show that the labels are reliable.
W5. In the main table, B4I uses interaction-level supervision while direct classifiers use item-level labels, so the gains may reflect the extra supervision rather than the neural-logical + aggregation design.
Missing comparisons include
- vanilla FENCR + the same Stage II
- simple score statistics (mean/quantile/histogram) + LR/MLP
- an interaction-pretrained model with the same aggregator
- DeepSets/MIL on raw features
- rule-only vs. scores-only ablations.
W6. Each item requires 155 or 2,048 surrogate calls, but
- latency
- memory
- parameter count
- the panel-size trade-off are not reported.
The data pipeline is also incomplete:
- raw sample sizes
- filtering criteria
- class distribution
- vocabulary fitting scope
- semantic attribute extraction
- time windows
should be described.
W7. The paper states that supports a higher interaction score and opposes it, but also says that is a learned combination rather than an unweighted sum of contributions. If is a general nonlinear function, a positive does not necessarily raise the final output. The manuscript neither gives the formula for nor validates its sign.
An exact logit decomposition, a monotonicity constraint, an interventional definition, or rule-deletion experiments would be needed to support the claim that signed contributions faithfully explain predictions.
Questions for authors
See Weakness