- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors used reinforcement learning to improve upon CS-ARM-BN for test time adaptation in biomedicine by treating the layer norm parameters in different layers as different settings to be evaluated.
Critical strengths
The idea to use re-inforcement learning to improve upon test-time adaptation is to the best of my knowledge novel and interesting. Defining discrete sets to limit the search space for the re-inforcement learning is interesting and makes sense in the context of biomedicine.
Critical weaknesses
One critical weakness is the lack of care for the manuscipt:
- The paper is unfinished as the authors were extremely wasteful with the usage of space (Line 291 is just p_src) 4 pages before the start of the method section. Still, the authors do not manage to even fill up the 9 pages and are missing even basic information such as which dataset they used to train/evaluate their model. They are also missing mandatory section such as the AI Use statement.
- The paper is also missing citations (the entire related work section has multiple ?? instead of citations), which stem from errors in the latex.
- The paper also lacks any description of their implementation details and examples of the data used and any quantitativ results.
- Their recomputed values lack any standard deviation, while all other values have them.
- Lack of comparison methods. They compare themselves with a single method on a single dataset and use the method as a fallback in case the performance did not increase.
Another critical weakness is the general assumption that using non-pooled statistics is better than pooled statistics. The authors did not show that there even is a configuration that outperforms just using pooled statistics everywhere. The final result is marginally better than just using CS-ARM-BN and no upper bound is provided.
Questions for authors
The manuscript would need to be completely rewritten for me to materially change my opinion, which is not the aim of the rebuttal.