- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents ICL-Surg for few-shot surgical phase recognition from a single annotated demonstration video. A frozen surgical VLP encoder feeds a GRU local branch and a max-pooled global branch; query windows are classified by fused cosine similarity to demonstration prototypes and then refined by Viterbi decoding using transition and positional priors from both the demonstration and a label pool. Experiments on four held-out surgical datasets report large frame-accuracy gains over several few-shot baselines, with ablations for the fusion and refinement modules.
Critical strengths
- Practical setting and simple modular design. The method requires only one demonstration video and no test-time parameter updates. The ablation isolates DCF and WLR, with WLR contributing +8.65 to +12.85 points, supporting the central claim that workflow priors can compensate for scarce annotations.
- Consistent gains across procedure types. Evaluation across four target datasets suggests potential applicability to different surgical workflows.
Critical weaknesses
- The empirical comparison does not convincingly establish sota performance. It is unclear whether the Table 1 baselines use the same configuration and one-video evaluation protocol. The closest few-shot surgical baselines, such as SPA and the prototype-based single video method cited in the introduction, are not compared. Only frame accuracy is reported, without variance in Table 1 or per-phase precision/recall/F1. This matters because the gain over the PeskaVLP-only baseline (61.9 → 68.14 on Cholec80) is much smaller than the headline gain over generic baselines and may partly reflect backbone or protocol differences.
- The workflow-prior construction and inference protocol are under-specified and partly inconsistent. It is unclear how the empirical transition matrix and positional priors are estimated from a label pool spanning procedures with different phase inventories, whether classes are aligned, and whether any target-dataset videos contribute. Sections 3.3–3.4 describe class prototypes, while Section 3.4 inference assigns labels from all demonstration windows; the global feature is described both as cumulative per-window max pooling and as a single video-level vector. WLR is also non-causal and requires the full video, limiting online clinical use. These issues affect reproducibility and the validity of the reported gains.
- WLR constructs empirical transition and positional priors using datasets with different phase definitions and class counts (5–12). The paper does not explain how these priors are aligned with the held-out procedure's phase vocabulary. This is particularly concerning because WLR accounts for much of the reported improvement. Furthermore, BernBypass and StrasBypass represent the same procedure at different institutions, weakening the claim of evaluation on entirely unseen procedures.
- The method appears to use all annotated windows from a demonstration video at inference (Section 3.4), making the claimed ≤5-shot comparison in Table 2 difficult to interpret. Baselines are not clearly evaluated under equivalent annotation budgets and training conditions. Moreover, ICL-Surg underperforms HyperVLP on two of four datasets, and the closely related Surgical Phase Anywhere (SPA) method is not directly compared.
Questions for authors
- Were all Table 1 baselines trained and evaluated with the same frozen PeskaVLP encoder, episode construction, and single-demonstration protocol? Please report per-dataset standard deviations and per-phase metrics, not only frame accuracy.
- What exactly is the label pool used for the empirical transition matrix and positional priors in the leave-one-dataset-out setting? How are phase classes aligned across datasets with 5–12 different phases, and is any video from the held-out target dataset used to estimate these priors?
- At inference, do you use class prototypes or all windows of the demonstration video? Is the global feature the cumulative max at each window or a single video-level vector, and how are global prototypes formed?
- Since WLR is offline, what is the accuracy under causal or online decoding? Please ablate demonstration-only, empirical-only, and fused priors, and report sensitivity to λ and λ_pos.
- Why is confidence defined as std(similarities) × max(similarity), and is α_j class-specific? Cosine similarities can be negative; how is α kept stable? A comparison with a learned scalar gate would help isolate DCF’s benefit.
- Why are SPA and the single-annotated-video prototype method not included as baselines, given that they are the closest settings?
- How many annotated windows are actually used at inference, and how is the demonstration video selected? Were baselines evaluated using equivalent annotation budgets, pretrained representations, and dataset splits?
- In Equations (5)–(7), confidence appears class-dependent, but the fusion coefficient is not. How is this coefficient computed across classes, and how are potentially negative similarity-derived confidence values handled?
4 comments
Response to Reviewer MCmC(Q4–Q6)
Response to Q4:
We agree that the original WLR is an offline post-processing step since it performs Viterbi decoding over the entire target video. We have added a causal online variant and report both settings in the revision. In the online variant, each label is predicted using only the current and past observations without backtracking, and the positional normalization uses the demonstration length instead of the target length .
Table 5 reports the ablation over demonstration-only (demo), empirical-only (empirical) and fused priors under both offline and causal online decoding. Figure 4 reports the sensitivity to and on StrasBypass.
| Dataset | Prior | Offline | Online |
|---|---|---|---|
| Cholec80 | Demo | 70.49 0.75 | 55.96 0.56 |
| Empirical | 56.38 2.56 | 44.66 1.43 | |
| Fused | 68.14 1.53 | 51.21 0.89 | |
| StrasBypass | Demo | 66.98 1.77 | 45.30 1.38 |
| Empirical | 66.52 2.85 | 47.61 3.01 | |
| Fused | 70.84 1.30 | 45.62 1.66 |
| Sweep | Range | Peak accuracy |
|---|---|---|
| () | 0–1, step 0.2 | 70.84 at |
| () | 0–1, step 0.2 | 70.84 at |
Accuracy varies by less than one percentage point for and by about one percentage point for and the default configuration (, ) lies within this flat region, indicating that the results are not sensitive to the exact choice of the fusion weights.
The online accuracy is lower than the offline one because the two settings solve different decoding problems. Offline WLR finds the globally optimal phase sequence over the entire video via Viterbi backtracking which can revise early errors once future observations become available. Online decoding, in contrast, must emit each label as soon as the current window is processed so it solves a sequence of per-step problems and can only retain the locally optimal state at each step; early errors cannot be corrected by later evidence. On top of this, the online variant cannot access the true target length and replaces it with in the positional prior which introduces an additional mismatch.
Response to Q5:
The confidence is defined as , where std measures the dispersion of the query window's similarities across class prototypes and max measures the peak similarity to the best-matching class, so their product requires both a discriminative distribution and a high-confidence peak. is a per-sample scalar, not per-class. Eq. (5) describes as "the set of support sample indices for the -class", which can be misread as per-class. In the implementation, std and max are computed over the similarity vector of all class prototypes, yielding one per query window, and the revision will correct the notation in Eq. (5). Regarding stability under negative cosine similarities, the implementation maps similarities to via the affine transform before computing the confidences, so the confidences are non-negative and . The fusion itself still uses the original cosine similarities, while the mapping is used only for confidence computation.
With this mapping, the DCF ablation results (mean std over 5 runs) are:
| Dataset | std$\times$max | max-only | entropy | learned scalar |
|---|---|---|---|---|
| Cholec80 | 59.49 2.30 | 59.14 2.58 | 59.04 2.89 | 55.20 2.11 |
| StrasBypass | 57.99 1.59 | 57.30 3.68 | 56.76 2.93 | 56.89 3.25 |
All three per-sample adaptive variants remain numerically stable and outperform or match the learned scalar, confirming that the affine mapping removes the negative-similarity issue without changing the behaviour of DCF. The revision documents this implementation detail in Table 4.
Response to Q6/W4:
We thank the reviewer for the suggestion to compare with this method. SPA (Yuan et al., MICCAI 2025) is a multimodal few-shot adaptation framework but its shots are image-level annotations whereas in ICL-Surg a shot is a video-level demonstration. SPA defines phases in natural language text, aligns visual-text embeddings, and relies on a task graph as an explicit temporal prior. Ours is a purely visual in-context few-shot method: prototypes are built from the visual windows of a demonstration video with no text phase definitions and no external task graph. The two therefore differ in shot granularity, input modality and prior dependency so accuracy gaps cannot be attributed to the method itself.
Response to Reviewer MCmC(Q7–Q8)
Response to Q7:
At inference we use all annotated windows of the demonstration video. For each class present in the demonstration, the local prototype is the average of the local features of all windows of that class and the global prototype is the average of the corresponding cumulative global features so the number of annotated windows per class equals the number of windows of that class in the demonstration and is variable, not fixed at 5. The demonstration video is selected automatically as the video with the most classes by default and can also be specified explicitly. All compared methods use the same demonstration video. Regarding the baselines, all methods use exactly the same training and test splits and are re-trained and evaluated under the same evaluation pipeline. In terms of annotation budget, ICL-Surg uses one complete demonstration video (≤5% annotation) while the baselines use a 10-shot setting (about 10% annotation) so ICL-Surg uses less annotation and the comparison is conservative in favor of the baselines. Pretrained representations are not necessarily identical since each baseline uses its own encoder or pretrained representation as part of its method; what is shared is the data split, episode construction, and evaluation protocol.
Response to Q8:
This question partially overlaps with our previous response on DCF. As explained there, is a per-sample scalar, not per-class; the notation in Eq. (5) can be misread as per-class, and the revision will correct it. Regarding stability under negative cosine similarities, as explained earlier, similarities are mapped to before computing the confidences, while the fusion still uses the original cosine similarities, so we do not repeat the details here. The rationale for and the comparison with a learned scalar gate have also been addressed in the previous response and are not repeated here.
Response to Reviewer MCmC (Additional Weaknesses)
In addition to the questions above, several points raised in the Critical Weaknesses are not covered by the question list. We address them separately below for completeness.
On BernBypass and StrasBypass (W3). We acknowledge that both subsets correspond to gastric bypass surgery, and we no longer describe them as entirely unseen procedures. As also noted in SPA (Yuan et al., 2025) and the MultiBypass140 paper (Lavanchy et al., 2024), the two subsets differ substantially in phase definitions and surgical protocols, creating a notable domain gap between the Bern and Strasbourg centers. We therefore treat them as two distinct institutions/domains rather than two copies of the same procedure. The revised manuscript distinguishes cross-procedure generalization (Cholec80, AutoLaparo) from cross-institution generalization (BernBypass, StrasBypass) and states clearly that the latter does not constitute evaluation on an entirely unseen procedure type.
On the comparison with HyperVLP and the meaning of “≤5-shot” (W4). We clarify the positioning of Table 2. Table 2 is not intended to claim overall superiority over all baselines; its purpose is to show that, under a stricter annotation budget (a single demonstration video, ≤5% annotation), ICL-Surg still achieves the best result on some datasets while remaining competitive on the others. We acknowledge that ICL-Surg underperforms HyperVLP on BernBypass (62.77 vs. 72.3) and AutoLaparo (58.72 vs. 60.1), and this comparison is not under an equivalent annotation budget: HyperVLP uses a 10-shot setting, while ICL-Surg uses a single demonstration video (≤5% annotation), as stated in Table 2.
The Shot column in Table 2 follows the same definition as in HyperVLP (Hu et al., 2026), from which the baseline numbers are taken: a shot denotes the fraction of annotated data relative to the total dataset size. The “10” for the baselines therefore corresponds to 10% annotated data, and our “1 video (≤5)” means that the single demonstration video used as context accounts for no more than 5% of the annotated windows of the dataset. This is distinct from the K-shot used during training (K=5), which refers to the number of support windows sampled per class within each episode. At inference, the context is one complete demonstration video, and for each class present in the demonstration the prototype is the average over all annotated windows of that class; this is consistent with the ≤5 annotation budget in Table 2, because the entire demonstration is itself below 5% of the dataset, and the per-class window count is not capped at 5.
Response to Reviewer MCmC(Q1–Q3)
Thank you for your careful review. Our responses to each comment are given below:
Response to Q1/W1:
We clarify the evaluation protocol of Table 1. All methods in Table 1 use exactly the same training and test splits and are re-trained and evaluated under the same evaluation pipeline rather than using the data splits or configurations from their original papers. We note that the encoders or pretrained representations of the baselines are not necessarily identical since they are part of each method; what is shared is the training set, the test set, the episode construction, and the single-demonstration evaluation protocol. ICL-Surg results are averaged over 5 independent runs while each baseline was run once due to the high computational cost of re-training and re-testing. The variance in Table 1 is therefore meaningful only for ICL-Surg.Per-phase precision, recall, and F1 for all four test datasets have been added in Appendix C. The per-phase results show that phases with very few windows such as phases 6, 8, and 10 on BernBypass (94, 237, and 130 windows) and phase 4 on AutoLaparo (174 windows) obtain low F1 scores reflecting the difficulty of recognizing rare phases from a single demonstration.
Response to Q2/W2:
In the leave-one-dataset-out setting, the label pool used for the empirical transition matrix and positional priors consists of all video labels from the six datasets other than the target dataset; no video or label from the held-out target dataset is used to estimate these priors. We do not perform semantic label alignment across datasets. The phase labels in each dataset follow the official phase definitions and are numbered in workflow order so the labels are ordinal indices (0, 1, 2, …) rather than semantic phase IDs. The empirical transition matrix and positional priors are therefore estimated directly in ordinal space. The different class counts across datasets (5-12) do not affect this estimation: the priors capture coarse-grained workflow structure shared across procedures such as ordered progression, the relative position of the k-th phase, and permissible ordinal transitions rather than a semantic mapping between different phase vocabularies.
Response to Q3:
At inference, we use all annotated windows of the demonstration video but average them per class into a single prototype for each class, and match against prototypes rather than treating each demonstration window as an independent support sample and comparing it with each query window. Specifically, for each class present in the demonstration video, the local prototype is the average of the local features of all windows of that class, and the global prototype is the average of the corresponding cumulative global features, so each class has one local prototype and one global prototype. Regarding the global feature, Eq. (3) defines gt=max(gt−1,ht), where ht is the GRU output of the t-th window. The global feature is updated after each window. gTv is simply the value of this recursion at t=Tv, obtained after processing all windows, so it is not a final global feature shared by all query windows. Global prototypes are formed by averaging the cumulative global features gj of the windows belonging to each class in the demonstration video.