- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
In this paper, the authors present a method to add an optical element to an event camera to either motion stabilize or induce motion in the scene, in an attempt to produce an "optimal" event rate. The biological inspiration is the saccade.
This is achieved by providing a policy network with fixed-period buffered event frames. A CNN then proposes the magnitude of motion required. An IWE (image of warped events) is used to evaluate the outcome by measuring event occupancy.
Critical strengths
The paper is quite well presented in that it is easy to follow along what the authors are doing and why. There has been a good attempt at ablation and comparison to SoTA. The literature review seems pretty thorough to me, did not find much missing there.
Critical weaknesses
- I find the premise of this paper to be flawed. In the first diagram, the authors differentiate between scenarios of “too sparse” or “too noisy”. In fact, event sensors sample the scene at an optimal rate, that is, proportional to the change occurring in the scene. Trying to artificially control the events produced by the scene through relative movement of the sensor, or through stabilization thus defeats the point of using an event sensor in the first place. In this case, why not just use a standard video camera? Or better, a hybrid event sensor/standard camera.
- EIRM is not a generic framework for event rate control, since it assumes brightness consistency, which is not all the case in many everyday scenes. How would EIRM react to a sudden brightness change caused by lens flare, or a glint from a passing car, for example?
- EIRM seems to be somewhat chicken-and-egg, since the observation window (control interval) is fixed length - thus the problem is not really solved for that interval which could still suffer from little data or too much.
- In “Event-based motion segmentation by motion compensation” the authors of that paper discuss how relative motion impacts the integration time required for an IWE to be formed. I think that this might be interesting for the authors to consider, and should be cited here.
- The comparison to benchmarks seems a bit pointless to be honest. The comparison is just to event sensor simulators and thus the results on the arbitrary evaluation metrics are meaningless. I feel like the only place to appropriately include these works in a benchmark, would be to compare simulation fidelity.
- The method repeatedly describes a three-action control space:
- A_p={0,180,360}\text{ RPM}, but, Appendix D.3, page 17, states “For the controlled optical-policy ablation reported in the main paper, the evaluated checkpoint uses the restricted two-action set 360 RPM.” What is going on there?
- It’s not clear to me what role if any the IMU plays in this method. Because the motions in Blender are very synthetic sinusoids, so I do not think the model will learn anything about real world camera motions.
- I know that it is a big ask, but this paper would have been way better with some sort of physical prototype to prove out the idea in the real world, on some qualitative data. It is nearly impossible for me to believe that this approach, trained an evaluatied only on synthetic data, would actually work. At least not without seeing some real world demo.
- TEO (Eq 3-4) seem to not penalize background event redundancy. i.e two streams could have TEO=1 and one stream could have a bunch of background crap.
- Some of the math here seems just gratuitous. Especially Eq 8-10, those equations add no value in terms of formalizing the problem and would be way clearer with just a sentence in plain English.
Questions for authors
Som small suggestions:
- In “Adaptive Prism Excitation” the authors seem to have removed spaces before periods? Perhaps in an effort to reduce space, but it does make it distracting to read.
- What are “dwell constraints”? If not important, maybe just write “Controller parameters can be found in the Appendix”/