- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper tries to develop active selection methods for neural operator learning, targeting the setting in which there is significant distribution shift. The paper proposes the ConsensusCover method, in which unsolved candidate inputs are ranked based on how much that input would reduce the variance in a linear weighted Fourier model, and a kernel based on the Jacobian of the current neural operator. It counts how many agree in a top-k for each batch, and incorporates that into the selection. It works well on 5 out of 6 benchmarks, except Darcy / FNO.
Critical strengths
- The paper does a reasonable job of explaining the motivation of the problem, in which PDE solvers are costly, so its a good idea to train neural operators with as little labeled data as possible via active learning. Also it does a good job of defining two fairly sensible strategies of picking source inputs that are close to the deployable inputs, and pick inputs that are most informative for learning, and figures out a way of combining the strengths of both.
Critical weaknesses
- There is a severe lack of related work, with only 10 citations throughout the entire paper. The authors need to add related work, of which there is plenty (Multi-Resolution Active Learning of Fourier Neural Operators, Active learning for neural PDE solvers, Active learning with selective time-step acquisition for PDEs, On the benefits of active data collection in operator learning, PICore: Physics-informed unsupervised coreset selection for data efficient neural operator training, etc.) to be able to show the positioning and novelty of the work in relation to the prior work.
- As a consequence of the severe lack of the related work, the baselines are quite weak.
Questions for authors
- How does prior work that leverages the geometry of the input space and the physics-informed nature of the problem to do active selection compare to ConsensusCover?
- What about standard active learning approaches, or even a random baseline?
- Can the authors explain the drop in performance for Darcy/FNO