- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies how to adapt a behavioral authentication model when user behavior changes, using only forward passes. DACS uses ablations on development data to select four model components, and then it searches for a separate scaling factor for each component using current-user data. An update is applied only if it improves BA on a separate validation split. With 20 objective evaluations, the paper reports BA gains of 3.037 percentage points on H-MOG and 1.646 on HuMI. The experiments also examine component selection and update directions, a discrete search version, and a possible application to a vision task.
Critical strengths
-
I like how little has to change in the model. You just rescale existing components, so there's no extra computation at inference. For k = 8, even the simpler 17-evaluation search keeps about 77–80% of the gain from the 20-evaluation GP search, without have to fit a GP.
-
The Fisher control in Table 3 is so useful. ComAtt is about 0.4pp ahead on both datasets. Although it is not a huge jump, but the difference holds up in the paired user-level tests. The direction comparison is more interesting to me: with ComAtt, letting components scale down as well as up gives about 1.59 points on H-MOG and 1.27 on HuMI over positive-only updates.
Critical weaknesses
-
I'm not quite convinced by the conclusion about historical directions. HMGSS has just one scalar for moving along a historical weight difference, while DACS can adjust four selected components separately. So there is two things change here, including the information source and the freedom to update the model. Another point is that I couldn't get from Supplementary Section 2.1 whether HMGSS used the same validation gate. This could be a limitation of the particular HMGSS setup, so my veiwpoint is that it is difficult to draw a general conclusion from its failure.
-
I am wodering that what happens to false accepts after adaptation. BA can go up even when FAR gets worse, e.g., if genuine-user acceptance improves enough to offset it. Figure 2 is only a schematic, and the actual FAR and FRR numbers aren't reported. Just keeping the threshold fixed doesn't guarantee the same FAR once the model changes. Without those numbers, I can't really judge the claim that impostor rejection is preserved.
Questions for authors
-
What checkpoints define the historical trajectory of HMGSS? Does it use the same search objectives, "no-update" options, evaluation budgets, and verification gating mechanisms as DACS?
-
Could you give more about the False Acceptance Rate (FAR) and False Rejection Rate (FRR), just like Static and DACS on Day 5. i think you can use a fixed threshold? Furthermore, please clarify whether the impostor samples used during the testing phase are distinct from those used during the search and verification phases. THis would help in assessing the practical trade-offs in authentication performance.