- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
To address challenges in federated test-time adaptation under continuous distribution shifts,specifically, when a client should recover, which clients to consult for assistance, and how many parameters to restore, the authors propose a recovery framework called FedART. FedART monitors risk using uncertainty signals calibrated against the source domain; upon triggering an alert, it retrieves client states that are similar, recent, and estimated to be reliable. It then selects recovery layers based on prediction concentration and employs a peer-prediction consensus mechanism to compare candidates across normal adaptation, local recovery, collaborative recovery, and rollback strategies. In my view, this work effectively resolves the robustness issues inherent in TTA-FedDG through the clever use of specific technical strategies.
Critical strengths
-
The author addresses a practical and complex problem, focusing on mechanisms of degradation and recovery during long-term test adaptation, while considering multidimensional factors such as client-side asynchronous drift, error accumulation, and communication overhead. The study specifically investigates "when to collaborate" and "how much to recover."
-
The authors provide a comprehensive treatment involving both experiments and theoretical analysis, accounting for various factors such as client heterogeneity, availability, population size, and state staleness; this demonstrates the system's robustness to a certain extent. Furthermore, the most notable feature of this work is its extremely low communication cost.
Critical weaknesses
1.Although the authors report average accuracy and worst-client accuracy, it remains difficult to characterize short-term performance collapses and recovery capabilities during the adaptation process. To demonstrate the method's robustness, the authors should provide additional metrics such as the miss rate, worst-case time-window accuracy, frequency of performance collapses, recovery time following degradation, and the proportion of recovery operations that actually improve the target accuracy.
2.The authors proposed this method to address the poor robustness associated with the ResNet model architecture; they should also provide results demonstrating the method's performance on pre-trained ViT models.
Questions for authors
- The author's figures need to be redrawn; some of the text and graphics appear flattened, which affects readability.
2.Refer to the issues mentioned in the weakness. I will lower the score if the author does not address my questions.