- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents EndoTME, a video enhancement framework designed for endoscopic scenes with spatially and temporally varying degradations. The method contains two main components. Reliable Trajectory Memory Retrieval (RTMR) traces historical features along motion trajectories but, rather than aggregating all aligned observations, ranks them using appearance similarity, local quality, flow reliability, and degradation risk before retaining the top memories. Physics-Inspired Degradation Conditioning (PIDC) constructs analytical cue maps for red veiling, specular reflection, blur, weak contrast, structural response, and motion, and uses them at both global and spatial levels to condition restoration and route features through three restoration experts. The method is evaluated on the paired STSVD dataset and two real-world datasets, LSVD and the internally collected EndoReal dataset. EndoTME reports the best results among the compared methods on the paired full-reference metrics and on the reported no-reference and temporal metrics for the real datasets. The paper also includes module-level and internal ablations for RTMR and PIDC, a Top-K sensitivity analysis, qualitative visualizations, and a runtime/parameter comparison.
Critical strengths
A main strength of the paper is the distinction between temporal correspondence and temporal reliability. In endoscopic videos, an aligned historical observation can still be severely corrupted by blood, reflection, blur, or motion-estimation error. RTMR explicitly evaluates historical candidates before aggregation rather than assuming that successful motion alignment makes them useful. The internal ablation supports this motivation: progressively adding feature similarity, local quality, flow reliability, and degradation risk improves the overall restoration and temporal metrics. The experimental study is also reasonably broad. The authors evaluate on a paired benchmark as well as two real-world datasets, including a relatively large internally collected dataset with patient-disjoint splitting. The factorial ablation of RTMR and PIDC is useful, and the reported performance is not obtained at the cost of a particularly large or slow model: EndoTME has 10.8M parameters and reports a favorable runtime–accuracy trade-off.
Critical weaknesses
My first concern is that the evidence for the broad multi-degradation restoration claim is somewhat indirect. The full-reference benchmark, STSVD, is primarily a smoky–clean dataset, whereas the more diverse degradations motivating PIDC, including blood-related veiling, reflection, blur, and low contrast, are mainly present in the real-world datasets. These datasets have no aligned clean targets and are evaluated after dataset-specific reference-free adaptation using no-reference IQA and temporal metrics. These results are useful, but they do not directly establish that the different degradation types are faithfully restored rather than simply producing outputs favored by the no-reference metrics. This is particularly relevant for medical video enhancement, where anatomical distortion or removal of clinically meaningful structures is more consequential than perceptual appearance alone. My second concern is that the temporal evaluation is not fully independent of the temporal machinery used by the method. EndoTME relies on optical flow for trajectory construction, includes an optical-flow-based temporal consistency loss during training/adaptation, and evaluates temporal quality using tWE and tOF with the same flow-estimation setup. This may favor methods whose outputs are explicitly optimized to be consistent under that estimator. An evaluation with an independent flow estimator or a complementary flow-free temporal metric would make the temporal-consistency claim more convincing.
Questions for authors
Q1: Are all methods in Table 2 adapted on the same LSVD and EndoReal splits with comparable training budgets? If not, please clarify the adaptation protocol for each baseline and provide a fair matched-adaptation comparison.
Q2: RTMR, the temporal loss, and the tWE/tOF metrics all rely on optical flow. Please evaluate temporal consistency with an independent flow estimator or a flow-free temporal metric to rule out possible metric coupling.
Q3: STSVD mainly provides paired smoky-clean data, while PIDC targets several additional degradations, including blood veiling, specular reflection, blur, and low contrast. Please provide degradation-specific quantitative results for these cases.
Q4: The real-world evaluation relies mainly on no-reference image-quality and temporal metrics. Can the authors provide an anatomical or downstream-task evaluation, such as segmentation, tracking, landmark detection, or clinician assessment, to verify that enhancement does not alter clinically relevant structures?
Q5: How much of PIDC’s gain comes from the specific analytical degradation cues rather than from simply adding extra spatial conditioning? Please include a control using learned degradation features or shuffled/mismatched cue maps while keeping the routing architecture unchanged.
4 comments
Rebuttal to Reviewer CasN [part 2]
To Q4 and W1:
To address the reviewer's concern regarding the preservation of clinically relevant anatomical structures, we conducted an additional downstream semantic segmentation experiment on CholecSeg8k to assess whether enhanced images retain important structural information in surgical scenes. CholecSeg8k contains 8,080 annotated frames from 17 laparoscopic videos, covering 13 semantic classes with pixel-level annotations. We adopt a video-level split of CholecSeg8k, with 9 videos (3,520 frames) for training, 3 videos (1,920 frames) for validation, and 5 videos (2,640 frames) for testing. We employ SegFormer-B2, initialized with ImageNet-1K pretrained weights, as a unified segmentation model. To fairly assess the impact of different enhancement methods on downstream segmentation, the segmentation model is trained exclusively on raw images and then applied to both raw test images and the corresponding images enhanced by each method, without method-specific retraining. All methods share the same segmentation model and ground-truth annotations within each test split. We report mIoU and mDice to evaluate segmentation performance.
Table R2. Downstream semantic segmentation results on CholecSeg8k.
| Method | mIoU ↑ | mDice ↑ |
|---|---|---|
| Raw | 0.5042 | 0.5725 |
| RVRT | 0.6705 | 0.7831 |
| DiffUIR | 0.6826 | 0.8057 |
| SelfSVD | 0.5029 | 0.6372 |
| MedVSR | 0.3807 | 0.5114 |
| EndoIR | 0.6838 | 0.8046 |
| EndoTME | 0.6937 | 0.8158 |
EndoTME achieves the highest mIoU (0.6937) and mDice (0.8158) among all compared methods, improving upon raw images by 0.1895 and 0.2433, respectively. Under the same segmentation model and annotation protocol, these results indicate that EndoTME-enhanced images better support downstream semantic segmentation under a fixed segmentation model, providing complementary evidence of structural information preservation.We have added the downstream segmentation results to Appendix G.3 (Line 1255) of the revised manuscript.
To Q5:
We thank the reviewer for the valuable suggestion to further examine the contribution of analytical degradation cues beyond additional spatial conditioning. To address this concern, we consider two complementary controls: the Learnable Tokens variant reported in Table 5 and a newly introduced Mismatched Cue Maps experiment. The former replaces degradation-derived conditioning with learnable tokens, while the latter replaces the six analytical cue maps entering PIDC with maps from different videos during inference. In the mismatched-cue experiment, RTMR, the spatial routing architecture, the three restoration experts, and all model parameters remain unchanged.
Table R3. Ablation of analytical degradation cues.
| Configuration | PSNR ↑ | SSIM ↑ |
|---|---|---|
| Learnable Tokens | 24.71 | 0.8842 |
| Mismatched Cue Maps | 23.49 | 0.8734 |
| Full PIDC | 25.88 | 0.8922 |
Full PIDC outperforms Learnable Tokens by 1.17 dB in PSNR and 0.0080 in SSIM, demonstrating the benefit of degradation-derived conditioning over generic learnable tokens. Furthermore, mismatching the cue maps reduces PSNR by 2.39 dB and SSIM by 0.0188 compared with Full PIDC, despite using the same trained model and routing architecture. These results suggest that the correspondence between analytical cue maps and input image content is important for effective degradation-conditioned restoration.
We welcome further discussion if any concerns remain.
Response to author
Thank you for your response. I believe you have addressed most of my concerns; although a few questions still remain, I will give you a higher rating.
Response to CasN
Thank you for your recognition and support of our work and rebuttal. If you have any remaining questions or concerns, we would be happy to provide further clarification.
Rebuttal to Reviewer CasN [part 1]
We appreciate your valuable feedback and support of our work.
For clarity, we refer to the reviewer's first concern regarding multi-degradation evaluation and anatomical fidelity as W1, and the second concern regarding the independence of temporal evaluation as W2.
To Q1:
All methods in Table 2 use identical data splits and strictly matched adaptation budgets within each target dataset.
Data splits. LSVD follows the official 416/70 training/test split, with the 416 training clips further divided into 374 adaptation and 42 validation clips, while the official 70 test clips remain strictly held out. EndoReal is partitioned at the patient level into adaptation, validation, and held-out test sets, comprising 256, 32, and 32 patients and 4,481, 598, and 612 video clips, respectively.
Adaptation and training budgets. All six methods undergo target-domain adaptation separately on LSVD and EndoReal under matched optimization budgets. Specifically, all methods receive 16k optimizer updates on LSVD and 84k on EndoReal, with the same effective number of target frames per update. Each method retains its original architecture and input formulation, ensuring identical target-domain data access and matched training opportunities within each dataset. Checkpoint selection and evaluation. All methods select checkpoints using the corresponding validation sets, while the held-out test sets are reserved exclusively for final evaluation. All methods are evaluated on identical test sequences using the same preprocessing, evaluation scripts, and metric implementations.
Therefore, Table 2 already presents a fair matched-adaptation comparison under controlled data splits, optimization budgets, and evaluation protocols.
To Q2 and W2:
We thank the reviewer for raising the concern about potential dependence on optical-flow estimation in temporal consistency evaluation. We would like to clarify that our original experiments already employed an independent optical-flow estimator for temporal consistency evaluation. Specifically, SpyNet is used for trajectory construction in RTMR and for computing the temporal consistency loss, whereas tWE and tOF are computed using an independent GMFlow estimator. GMFlow is used exclusively for evaluation and is independent of the SpyNet estimator employed within the model. All compared methods are evaluated using the same GMFlow estimator and evaluation protocol to ensure a consistent and fair comparison.
To Q3 and W1:
To address the concern regarding limited degradation-specific quantitative evidence, we conducted additional comparisons on four degradation-specific subsets of the paired STSVD test data, covering Blood Veil, Specular Reflection, Motion Blur, and Low Contrast. All compared methods are evaluated on the same test samples using a unified evaluation protocol, with PSNR and SSIM reported for each degradation category.
Table R1. Degradation-specific quantitative comparison. Each entry reports PSNR (dB) ↑ / SSIM ↑.
EndoTME achieves the highest SSIM across all four degradation categories and the best PSNR in three categories: Blood Veil, Specular Reflection, and Motion Blur. For Low Contrast, EndoTME achieves the second-best PSNR while maintaining the highest SSIM. These degradation-specific full-reference results provide additional quantitative support for the effectiveness of our framework under diverse visual degradations.We will incorporate these results and the corresponding evaluation protocol into the revised manuscript.