- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
- This paper tried to solve the blind panoramic image super-resolution task with limited computation resources during training.
- Several key components are proposed to achieve this:
- Instead of using ERP representation, they use the six cubemap representation. To reduce the size of the gradient map, they randomly pick some of the faces to propagate the gradients instead of all faces. This is called Spherical Subgraph Routing (SSR).
- To reduce the hallucination of the diffusion, the also proposed Fourier Pseudo-inverse Guidance (FPG). It avoid the intermediate features deviate too much from the input low resolution features.
- To reduce the size of the attention map, they also proposed Spherical Octant Attention. The global attention among all 6 faces is huge. The proposed attention mechanism first split the cubemap into a one level octree with 8 sub-cubes, then groups the regions around the cubemap corners and each sub-face only attend to faces that share corners. This significantly reduced the attention map size.
- They conduct experiments on different datasets including ODISR and SUN360 using Real-ESRGAN degradation.
Critical strengths
- I think SSR is interesting, the separation of forward and backward pass to save memory is clever. As shown in the Table S2, K=1 reduces the memory usage by around 70% compared with K=6.
- Table 2 shows the good efficiency of the proposed method compared with other methods. Almost half of the second best OSEDiff.
- The performance is the best on most metrics as shown in the Table 1. Other metrics are also comparable. Honestly, the blue in the table is really close to black... Try a more distinguishable color.
- Ablation study in Table 3 shows the effectiveness of both SSR and FPG.
Critical weaknesses
- It is a bit weird that Table 4 only shows the performance of K=1,2,3,4. What about K=5,6. If this experiment is unable to run on the available devices, how can you get the memory usage of K=5,6? It is also anti-intuitive that the K=3 is better than higher K.
- I am thinking if FPG is correct. For complicated degradation, the assumption in Equation 4 is hard to get, and the degradation matrix is hard to approximate. Training-free FPG with strong guidance as shown in Table S3 also shows that there might be errors in the estimated degradation.
Questions for authors
- The entire paper emphasizes the "full context" concept, any evidence about the necessity of the full context?
- If you convert the ERP to cubemap then super resolve each view, how to completly avoid seams when you stitch them together by the end? Have you analyze the seams?
- I wonder if the proposed modules can be generalized to other architectures,
- I would like to see if FPG works on real degradations.