- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies sequential field deletion under a bound on deviation from the original prediction. Fields are ranked once by their individual prediction effects, then tested for deletion within descending cost tiers. Experiments on taxi and movie data compare retained cost and prediction error against lexical, random, and dynamically updated orders.
Critical strengths
- The question is specific. Field deletion is written as an audited procedure under a query budget, where each deletion is checked against the original prediction and the queries spent on scoring are counted.
- The paper reports the boundary of its claim. Prediction error stays within tolerance over the first four budgets, wider budgets save more but can exceed it, and a different masking of deleted fields can reverse the advantage.
Critical weaknesses
- The reported results do not establish an efficiency advantage of static scoring over dynamic updating. Dynamic updating lowers the retained cost in 18 of 20 conditions and uses 15 and 14 queries on NYC and IMDb against 16 and 14 for static ordering. The paper acknowledges this limitation, but does not establish under what conditions static scoring would offer a better trade-off than dynamic updating.
- The comparison also depends on a rule that forbids retries. Each field is tried once, and a rejected field is never tried again. The paper's own example shows the rule can decide the result: the lexical order rejects b, a second try would delete b, but the rule forbids it. Whether static ordering still wins when retries are allowed is not reported.
Questions for authors
- Since dynamic updating is no more expensive in the reported runs, under what conditions does computing the scores once do better?
- How often would a second try have succeeded in the reported runs? Would allowing retries change the comparison between orderings?
- Dynamic updating wins in 18 of 20 conditions. Which two conditions favour static ordering, and how large is the difference there?