- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces “deployment-time robustness routing,” which really is a mixture of techniques: (1) a way to produce a “model bank” of finetuned, specialized models from a single model, spanning two dimensions: specialization and “robustness”, (2) a way to combine models from this model bank at test time to make better predictions than using just the original single model checkpoint.
Critical strengths
It is interesting how Hierarchical Routing+ demonstrates complementary benefits of routing and local test-time adaptation.
Critical weaknesses
Model bank construction: A fair bit of the paper (and appendix) are devoted to discussing the “robustness” parameter lambda and the two-dimensional filter bank (specialization and robustness). However, this model bank is not compared to other approaches.
Questions for authors
Why this particular model bank design? Did you consider alternatives? Why do we need the robustness dimension? Do you have baselines without it?
Small detail: Please fix the Figure 2 legend - it is all the same shape.
2 comments
Thank you for your reply. I will be interested to see that explored more in-depth in future work. As it stands, I maintain my rating.
Author Response to Reviewer AEeN
We appreciate the reviewer's recognition of the complementary benefits of routing and test-time adaptation. Our model-bank design is motivated by the observation that different deployment distributions favor different robustness levels, as illustrated in Table 1. This justifies considering robustness alongside domain specialization. We agree that comparing our ERM/KL-DRO bank with an ERM-only bank would help quantify the contribution of the robustness dimension, and will consider this ablation in future work. We will also correct the Figure 2 legend.