- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LensErase, a model for removing objects and other content from images. It also constructs LensErase-100M, a dataset containing 101.07M samples covering objects, visual text, watermarks, and posters. The method is based on Lens and uses Mask-Aware RoPE to include mask information. Meanwhile, SFT, RL, and TraceFusion are used to improve the removal results and preserve unrelated content. Experiments on six public benchmarks and LensErase-Bench show that the method outperforms multiple baseline methods.
Critical strengths
- The dataset covers different removal tasks. The strategy of adding objects to real images and using the original images as clean targets is reasonable.
- The experiments cover six public benchmarks and 12 baseline methods. The results are strong. The ablation studies in Table 3 and Table 4 are also useful.
Critical weaknesses
- The paper uses 101.07M training samples and a large amount of computing resources. However, it does not report results with smaller training sets. Therefore, it is unclear how much of the improvement comes from the proposed method rather than the data scale.
- The RL training uses GPT-5.6 Sol to provide reward scores. Meanwhile, LensErase-Bench also uses the same model for evaluation. I am concerning whether there will be hacking with same reward model and judge model.
- In Appendix G, the Removal reward and removal-aware gating are removed together. Therefore, the independent contribution of the gating mechanism in Eq. (6) remains unclear.
Questions for authors
Please see the weaknesses above. I am willing to keep a positive rating if these concerns are properly addressed.
5 comments
2. Potential reward overoptimization
📝 Question. Can RL optimize preferences specific to GPT-5.6 Sol and thereby increase the LensErase-Bench score?
💡 Response. The RL reward combines VLM judgments with pixel fidelity terms, while LensErase‑Bench scores the full output image using a separate rubric. To verify that RL improves genuine removal quality without overfitting to GPT-5.6 Sol preferences, we conducted three independent checks outside the training judge:
1. SFT-to-RL gains across unseen VLM judges.
We re-evaluated the exact same SFT and RL (Raw) outputs (before TraceFusion) across all 500 LensErase-Bench cases using three unseen judges: GPT-6 Astra, GPT-6.1 Sol, and cross-family Gemini 3.8 Flash. All three independent judges report consistent gains in Overall Score and HiFi Removal, closely matching the +10.80 pp under GPT-5.6 Sol. These steady improvements confirm that RL genuinely resolves incomplete removals and local artifacts, without relying on judge-specific biases.
| Judge | SFT Overall | RL (Raw) Overall | Δ Overall | SFT HiFi Removal (%) | RL (Raw) HiFi Removal (%) | Δ HiFi Removal (pp) |
|---|---|---|---|---|---|---|
| GPT-5.6 Sol (paper) | 89.91 | 94.41 | +4.50 | 70.80 | 81.60 | +10.80 |
| GPT-6 Astra | 91.45 | 95.09 | +3.64 | 70.20 | 81.00 | +10.80 |
| GPT-6.1 Sol | 92.03 | 95.87 | +3.84 | 72.80 | 83.00 | +10.20 |
| Gemini 3.8 Flash | 92.31 | 96.69 | +4.38 | 79.40 | 90.00 | +10.60 |
2. Final method rankings across unseen VLM judges.
Using the exact same model outputs and scoring rubric as in Table 2, we re-evaluated LensErase and the six representative baselines across all 500 cases of LensErase-Bench with three independent VLM judges, including two newer models (GPT-6 Astra, GPT-6.1 Sol) and a cross-family judge (Gemini 3.8 Flash).
Across all four judges, LensErase consistently ranks 1st in both Overall Score and HiFi Removal ratio. Competing baselines also show only minor rank swaps among closely scored models, confirming the robustness of our evaluation rubric across VLM families.
| Method · Overall | GPT-5.6 Sol Score | GPT-5.6 Sol Rank | GPT-6 Astra Score | GPT-6 Astra Rank | GPT-6.1 Sol Score | GPT-6.1 Sol Rank | Gemini 3.8 Flash Score | Gemini 3.8 Flash Rank |
|---|---|---|---|---|---|---|---|---|
| LensErase (w/ TraceFusion) | 95.22 | 1 | 95.29 | 1 | 95.61 | 1 | 96.87 | 1 |
| OmniPaint | 73.59 | 2 | 73.91 | 4 | 75.04 | 4 | 77.46 | 5 |
| OSOR | 71.45 | 4 | 75.37 | 2 | 76.66 | 2 | 80.88 | 3 |
| ObjectClear | 71.67 | 3 | 74.06 | 3 | 76.26 | 3 | 81.18 | 2 |
| RORem-mixed | 69.75 | 5 | 72.63 | 5 | 74.35 | 5 | 79.96 | 4 |
| GeoRemover | 62.70 | 6 | 60.39 | 7 | 61.92 | 7 | 67.74 | 7 |
| OmniEraser | 58.89 | 7 | 63.97 | 6 | 65.93 | 6 | 69.72 | 6 |
| Method · HiFi Removal (%) | GPT-5.6 Sol Score | GPT-5.6 Sol Rank | GPT-6 Astra Score | GPT-6 Astra Rank | GPT-6.1 Sol Score | GPT-6.1 Sol Rank | Gemini 3.8 Flash Score | Gemini 3.8 Flash Rank |
|---|---|---|---|---|---|---|---|---|
| LensErase (w/ TraceFusion) | 82.80 | 1 | 81.00 | 1 | 83.60 | 1 | 90.20 | 1 |
| OmniPaint | 42.80 | 2 | 35.20 | 2 | 38.20 | 2 | 48.80 | 4 |
| OSOR | 32.60 | 3 | 33.20 | 3 | 36.20 | 3 | 54.40 | 2 |
| ObjectClear | 28.40 | 4 | 28.60 | 4 | 34.00 | 4 | 49.80 | 3 |
| RORem-mixed | 27.40 | 5 | 23.20 | 5 | 26.00 | 5 | 45.80 | 5 |
| GeoRemover | 25.60 | 6 | 20.80 | 6 | 25.40 | 6 | 37.40 | 6 |
| OmniEraser | 22.60 | 7 | 19.80 | 7 | 23.60 | 7 | 33.80 | 7 |
3. Blind human pairwise evaluation. We conducted a blind human pairwise study comparing LensErase with the six baselines in Table 2. Twenty participants each completed 100 comparisons (2,000 judgments in total). Given the marked input image and two anonymized outputs in randomized A/B order, participants chose the result with cleaner target and effect removal, more natural inpainting, and better background preservation (ties allowed).
Fitting a Bradley-Terry model [1] (ties as half-wins, centered at 1500 Elo), LensErase ranks 1st at 1770.5 Elo, leading the strongest baseline ObjectClear by 249.0 Elo with 465 wins, 47 ties, and 59 losses. The human ranking also aligns closely with the GPT-5.6 Sol rankings on LensErase-Bench.
[1] Bradley, Ralph Allan, and Milton E. Terry. "Rank analysis of incomplete block designs: I. the method of paired comparisons." Biometrika 39.3/4 (1952): 324-345.
| Method | Elo ↑ | Human Rank | Wins | Ties | Losses | VLM Overall Score Rank | VLM HiFi Removal Rank |
|---|---|---|---|---|---|---|---|
| LensErase | 1770.5 | 1 | 465 | 47 | 59 | 1 | 1 |
| ObjectClear | 1521.5 | 2 | 290 | 37 | 244 | 3 | 4 |
| OmniPaint | 1507.4 | 3 | 279 | 34 | 258 | 2 | 2 |
| OSOR | 1483.6 | 4 | 256 | 41 | 275 | 4 | 3 |
| RORem-mixed | 1462.5 | 5 | 239 | 38 | 295 | 5 | 5 |
| GeoRemover | 1410.6 | 6 | 202 | 22 | 347 | 6 | 6 |
| OmniEraser | 1343.9 | 7 | 143 | 33 | 396 | 7 | 7 |
Summary. Taken together, these three additional evaluations support improved removal quality of RL and show no signs of reward overoptimization specific to GPT-5.6 Sol.
3. Independent effect of the removal-aware gate
📝 Question. Does the ablation isolate the effect of the gate in Eq. (6)?
💡 Response. In Table 17 of the submission, w/o Removal and gate is a joint ablation. In Eq. (6), each zero-scored Removal subcriterion halves the aggregate RL reward. This gate is designed to prevent strong scores on other reward components (such as Preservation or Cleanliness) from offsetting clear removal failures. To isolate this gate, we trained an additional variant (w/o gate) that sets the gate multiplier to 1 while keeping all five reward components and weights unchanged, including Removal.
Comparing w/o Removal and gate , w/o gate , and Full reward separates the two effects. The Removal term provides the main gain (+38.00 pp), while the gate adds another +4.80 pp in HiFi Removal, +3.90 points in Removal, and +0.51 dB in mean PSNR across the six public benchmarks. Without the gate, RL can offset incomplete removals with higher Preservation and Cleanliness scores, which lowers the HiFi Removal rate.
| Model variant | Removal | Inpainting | Preservation | Cleanliness | Overall | HiFi Removal (%) | Public PSNR ↑ | Public LPIPS ↓ |
|---|---|---|---|---|---|---|---|---|
| Full reward (Raw) | 94.95 | 92.65 | 93.35 | 96.15 | 94.41 | 81.60 | 28.70 | 0.0816 |
| w/o gate | 91.05 | 89.10 | 97.15 | 97.40 | 93.15 | 76.80 | 28.19 | 0.0837 |
| w/o Removal and gate | 43.35 | 46.10 | 98.70 | 98.10 | 65.92 | 38.80 | 21.85 | 0.1446 |
| w/o Reconstruction | 90.85 | 91.40 | 96.10 | 95.70 | 92.98 | 78.00 | 28.32 | 0.0791 |
| w/o Preservation | 94.15 | 91.50 | 89.20 | 96.00 | 93.00 | 74.60 | 28.01 | 0.0817 |
| w/o Outside Fidelity | 92.90 | 91.70 | 94.40 | 96.40 | 93.66 | 78.00 | 26.69 | 0.0888 |
| w/o Cleanliness | 90.90 | 90.95 | 96.25 | 95.95 | 92.99 | 78.40 | 28.48 | 0.0803 |
We have incorporated these results and clarifications into the revised manuscript.
Reviewer 2esz
We sincerely thank Reviewer 2esz for the thoughtful assessment and constructive suggestions. Below, we address each point with a controlled reduced-data pretraining study (500K vs. 101.07M), multi-judge evaluations alongside a 2,000-judgment human study, and an isolated ablation of the removal-aware gate.
Please feel free to let us know if any further discussions or experiments would be helpful.
1. Contribution of pretraining data scale
📝 Question. How much of the improvement comes from the method and how much comes from the 101.07M pretraining examples?
💡 Response. Due to the high training cost (two weeks on 96 A100 GPUs) and the limited rebuttal timeframe, we cannot run additional data-scale sweeps from scratch. Instead, we report a controlled reduced-data experiment from our earlier development, trained on 500K sampled pretraining triplets (~0.5% of LensErase-100M). All architecture designs, SFT data (140K), and RL configurations (19K) were kept identical.
This comparison highlights two clear findings:
OmniPaint, scores 73.59 Overall and 42.80% HiFi Removal). Our 500K pipeline reaches 89.92 Overall and 71.20% HiFi Removal. Moreover, RL boosts HiFi Removal by +14.20 pp on the 500K model, confirming the effectiveness of our method.Pretraining on 101.07M triplets
Pretraining on 500K triplets