acceptodds
Under review as a conference paper at ICLR 2027

ICL-Surg: In-Context Learning with Logic Refinement for Surgical Phase Recognition

Abstract

Few-shot surgical phase recognition aims to segment surgical videos into semantically meaningful phases using only a few annotated samples. However, the scarcity of labeled data limits existing methods in capturing the temporal structure and discriminative patterns of surgical workflows. To address these challenges,we present ICL-Surg, a novel framework that combines in-context learning with workflow-guided logic refinement. Given a single annotated demonstration video,ICL-Surg extracts complementary local temporal and global scene representations and adaptively fuses them via a Dynamic Confidence-aware Fusion (DCF) mechanism, which balances local and global similarities according to modality confidence. For each segment in a query video, phase labels are assigned by weighted cosine similarity to the demonstration. To enforce temporal coherence and medical plausibility, we present a Workflow-guided Logic Refinement (WLR) module that applies Viterbi decoding with dual-source priors: a local transition matrix derived from the demonstration, and empirical transition and positional statistics aggregated from a label pool, combined by linear interpolation. This module explicitly refines the predicted phase sequence against surgical logic. Extensive experiments across multiple public surgical video benchmarks demonstrate that ICL-Surg outperforms existing few-shot surgical phase recognition methods.Code will be released upon acceptance.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
surgical phase recognition, meta learning, in-context learning, logic refinement

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_MCmCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents ICL-Surg for few-shot surgical phase recognition from a single annotated demonstration video. A frozen surgical VLP encoder feeds a GRU local branch and a max-pooled global branch; query windows are classified by fused cosine similarity to demonstration prototypes and then refined by Viterbi decoding using transition and positional priors from both the demonstration and a label pool. Experiments on four held-out surgical datasets report large frame-accuracy gains over several few-shot baselines, with ablations for the fusion and refinement modules.

Critical strengths
  • Practical setting and simple modular design. The method requires only one demonstration video and no test-time parameter updates. The ablation isolates DCF and WLR, with WLR contributing +8.65 to +12.85 points, supporting the central claim that workflow priors can compensate for scarce annotations.
  • Consistent gains across procedure types. Evaluation across four target datasets suggests potential applicability to different surgical workflows.
Critical weaknesses
  • The empirical comparison does not convincingly establish sota performance. It is unclear whether the Table 1 baselines use the same configuration and one-video evaluation protocol. The closest few-shot surgical baselines, such as SPA and the prototype-based single video method cited in the introduction, are not compared. Only frame accuracy is reported, without variance in Table 1 or per-phase precision/recall/F1. This matters because the gain over the PeskaVLP-only baseline (61.9 → 68.14 on Cholec80) is much smaller than the headline gain over generic baselines and may partly reflect backbone or protocol differences.
  • The workflow-prior construction and inference protocol are under-specified and partly inconsistent. It is unclear how the empirical transition matrix and positional priors are estimated from a label pool spanning procedures with different phase inventories, whether classes are aligned, and whether any target-dataset videos contribute. Sections 3.3–3.4 describe class prototypes, while Section 3.4 inference assigns labels from all demonstration windows; the global feature is described both as cumulative per-window max pooling and as a single video-level vector. WLR is also non-causal and requires the full video, limiting online clinical use. These issues affect reproducibility and the validity of the reported gains.
  • WLR constructs empirical transition and positional priors using datasets with different phase definitions and class counts (5–12). The paper does not explain how these priors are aligned with the held-out procedure's phase vocabulary. This is particularly concerning because WLR accounts for much of the reported improvement. Furthermore, BernBypass and StrasBypass represent the same procedure at different institutions, weakening the claim of evaluation on entirely unseen procedures.
  • The method appears to use all annotated windows from a demonstration video at inference (Section 3.4), making the claimed ≤5-shot comparison in Table 2 difficult to interpret. Baselines are not clearly evaluated under equivalent annotation budgets and training conditions. Moreover, ICL-Surg underperforms HyperVLP on two of four datasets, and the closely related Surgical Phase Anywhere (SPA) method is not directly compared.
Questions for authors
  • Were all Table 1 baselines trained and evaluated with the same frozen PeskaVLP encoder, episode construction, and single-demonstration protocol? Please report per-dataset standard deviations and per-phase metrics, not only frame accuracy.
  • What exactly is the label pool used for the empirical transition matrix and positional priors in the leave-one-dataset-out setting? How are phase classes aligned across datasets with 5–12 different phases, and is any video from the held-out target dataset used to estimate these priors?
  • At inference, do you use class prototypes or all windows of the demonstration video? Is the global feature the cumulative max at each window or a single video-level vector, and how are global prototypes formed?
  • Since WLR is offline, what is the accuracy under causal or online decoding? Please ablate demonstration-only, empirical-only, and fused priors, and report sensitivity to λ and λ_pos.
  • Why is confidence defined as std(similarities) × max(similarity), and is α_j class-specific? Cosine similarities can be negative; how is α kept stable? A comparison with a learned scalar gate would help isolate DCF’s benefit.
  • Why are SPA and the single-annotated-video prototype method not included as baselines, given that they are the closest settings?
  • How many annotated windows are actually used at inference, and how is the demonstration video selected? Were baselines evaluated using equivalent annotation budgets, pretrained representations, and dataset splits?
  • In Equations (5)–(7), confidence appears class-dependent, but the fusion coefficient is not. How is this coefficient computed across classes, and how are potentially negative similarity-derived confidence values handled?
4 comments
AuthorsOpenReview ↗

Response to Reviewer MCmC(Q1–Q3)

Thank you for your careful review. Our responses to each comment are given below:

Response to Q1/W1:

We clarify the evaluation protocol of Table 1. All methods in Table 1 use exactly the same training and test splits and are re-trained and evaluated under the same evaluation pipeline rather than using the data splits or configurations from their original papers. We note that the encoders or pretrained representations of the baselines are not necessarily identical since they are part of each method; what is shared is the training set, the test set, the episode construction, and the single-demonstration evaluation protocol. ICL-Surg results are averaged over 5 independent runs while each baseline was run once due to the high computational cost of re-training and re-testing. The variance in Table 1 is therefore meaningful only for ICL-Surg.Per-phase precision, recall, and F1 for all four test datasets have been added in Appendix C. The per-phase results show that phases with very few windows such as phases 6, 8, and 10 on BernBypass (94, 237, and 130 windows) and phase 4 on AutoLaparo (174 windows) obtain low F1 scores reflecting the difficulty of recognizing rare phases from a single demonstration.

Response to Q2/W2:

In the leave-one-dataset-out setting, the label pool used for the empirical transition matrix and positional priors consists of all video labels from the six datasets other than the target dataset; no video or label from the held-out target dataset is used to estimate these priors. We do not perform semantic label alignment across datasets. The phase labels in each dataset follow the official phase definitions and are numbered in workflow order so the labels are ordinal indices (0, 1, 2, …) rather than semantic phase IDs. The empirical transition matrix and positional priors are therefore estimated directly in ordinal space. The different class counts across datasets (5-12) do not affect this estimation: the priors capture coarse-grained workflow structure shared across procedures such as ordered progression, the relative position of the k-th phase, and permissible ordinal transitions rather than a semantic mapping between different phase vocabularies.

Response to Q3:

At inference, we use all annotated windows of the demonstration video but average them per class into a single prototype for each class, and match against prototypes rather than treating each demonstration window as an independent support sample and comparing it with each query window. Specifically, for each class present in the demonstration video, the local prototype is the average of the local features of all windows of that class, and the global prototype is the average of the corresponding cumulative global features, so each class has one local prototype and one global prototype. Regarding the global feature, Eq. (3) defines gt=max⁡(gt−1,ht)g_t=\max(g_{t-1}, h_t), where hth_t is the GRU output of the tt-th window. The global feature is updated after each window. gTvg_{T_v} is simply the value of this recursion at t=Tvt=T_v, obtained after processing all windows, so it is not a final global feature shared by all query windows. Global prototypes are formed by averaging the cumulative global features gjg_j of the windows belonging to each class in the demonstration video.

AuthorsOpenReview ↗

Response to Reviewer MCmC(Q4–Q6)

Response to Q4:

We agree that the original WLR is an offline post-processing step since it performs Viterbi decoding over the entire target video. We have added a causal online variant and report both settings in the revision. In the online variant, each label is predicted using only the current and past observations without backtracking, and the positional normalization uses the demonstration length TdemoT_{\text{demo}} instead of the target length TT.

Table 5 reports the ablation over demonstration-only (demo), empirical-only (empirical) and fused priors under both offline and causal online decoding. Figure 4 reports the sensitivity to λ\lambda and λpos\lambda_{\text{pos}} on StrasBypass.

DatasetPriorOfflineOnline
Cholec80Demo70.49 ±\pm 0.7555.96 ±\pm 0.56
Empirical56.38 ±\pm 2.5644.66 ±\pm 1.43
Fused68.14 ±\pm 1.5351.21 ±\pm 0.89
StrasBypassDemo66.98 ±\pm 1.7745.30 ±\pm 1.38
Empirical66.52 ±\pm 2.8547.61 ±\pm 3.01
Fused70.84 ±\pm 1.3045.62 ±\pm 1.66
SweepRangePeak accuracy
λ\lambda (λpos=0.5\lambda_{\text{pos}}=0.5)0–1, step 0.270.84 at λ=0.6\lambda=0.6
λpos\lambda_{\text{pos}} (λ=0.6\lambda=0.6)0–1, step 0.270.84 at λpos=0.5\lambda_{\text{pos}}=0.5

Accuracy varies by less than one percentage point for λ∈[0,1]\lambda\in[0,1] and by about one percentage point for λpos∈[0.5,0.8]\lambda_{\text{pos}}\in[0.5,0.8] and the default configuration (λ=0.6\lambda=0.6, λpos=0.5\lambda_{\text{pos}}=0.5) lies within this flat region, indicating that the results are not sensitive to the exact choice of the fusion weights.

The online accuracy is lower than the offline one because the two settings solve different decoding problems. Offline WLR finds the globally optimal phase sequence over the entire video via Viterbi backtracking which can revise early errors once future observations become available. Online decoding, in contrast, must emit each label as soon as the current window is processed so it solves a sequence of per-step problems and can only retain the locally optimal state at each step; early errors cannot be corrected by later evidence. On top of this, the online variant cannot access the true target length TT and replaces it with TdemoT_{\text{demo}} in the positional prior which introduces an additional mismatch.

Response to Q5:

The confidence is defined as std(similarity)×max⁡(similarity)\mathrm{std}(\text{similarity}) \times \max(\text{similarity}), where std measures the dispersion of the query window's similarities across class prototypes and max measures the peak similarity to the best-matching class, so their product requires both a discriminative distribution and a high-confidence peak. αj\alpha_j is a per-sample scalar, not per-class. Eq. (5) describes Ic\mathcal{I}_c as "the set of support sample indices for the cc-class", which can be misread as per-class. In the implementation, std and max are computed over the similarity vector of all class prototypes, yielding one αj\alpha_j per query window, and the revision will correct the notation in Eq. (5). Regarding α\alpha stability under negative cosine similarities, the implementation maps similarities to [0,1][0,1] via the affine transform (cos⁡(⋅)+1)/2(\cos(\cdot)+1)/2 before computing the confidences, so the confidences are non-negative and αj∈[0,1]\alpha_j\in[0,1]. The fusion itself still uses the original cosine similarities, while the mapping is used only for confidence computation.

With this mapping, the DCF ablation results (mean ±\pm std over 5 runs) are:

Datasetstd$\times$maxmax-onlyentropylearned scalar
Cholec8059.49 ±\pm 2.3059.14 ±\pm 2.5859.04 ±\pm 2.8955.20 ±\pm 2.11
StrasBypass57.99 ±\pm 1.5957.30 ±\pm 3.6856.76 ±\pm 2.9356.89 ±\pm 3.25

All three per-sample adaptive variants remain numerically stable and outperform or match the learned scalar, confirming that the affine mapping removes the negative-similarity issue without changing the behaviour of DCF. The revision documents this implementation detail in Table 4.

Response to Q6/W4:

We thank the reviewer for the suggestion to compare with this method. SPA (Yuan et al., MICCAI 2025) is a multimodal few-shot adaptation framework but its shots are image-level annotations whereas in ICL-Surg a shot is a video-level demonstration. SPA defines phases in natural language text, aligns visual-text embeddings, and relies on a task graph as an explicit temporal prior. Ours is a purely visual in-context few-shot method: prototypes are built from the visual windows of a demonstration video with no text phase definitions and no external task graph. The two therefore differ in shot granularity, input modality and prior dependency so accuracy gaps cannot be attributed to the method itself.

AuthorsOpenReview ↗

Response to Reviewer MCmC(Q7–Q8)

Response to Q7:

At inference we use all annotated windows of the demonstration video. For each class present in the demonstration, the local prototype is the average of the local features of all windows of that class and the global prototype is the average of the corresponding cumulative global features so the number of annotated windows per class equals the number of windows of that class in the demonstration and is variable, not fixed at 5. The demonstration video is selected automatically as the video with the most classes by default and can also be specified explicitly. All compared methods use the same demonstration video. Regarding the baselines, all methods use exactly the same training and test splits and are re-trained and evaluated under the same evaluation pipeline. In terms of annotation budget, ICL-Surg uses one complete demonstration video (≤5% annotation) while the baselines use a 10-shot setting (about 10% annotation) so ICL-Surg uses less annotation and the comparison is conservative in favor of the baselines. Pretrained representations are not necessarily identical since each baseline uses its own encoder or pretrained representation as part of its method; what is shared is the data split, episode construction, and evaluation protocol.

Response to Q8:

This question partially overlaps with our previous response on DCF. As explained there, αj\alpha_j is a per-sample scalar, not per-class; the Ic\mathcal{I}_c notation in Eq. (5) can be misread as per-class, and the revision will correct it. Regarding αj\alpha_j stability under negative cosine similarities, as explained earlier, similarities are mapped to [0,1][0,1] before computing the confidences, while the fusion still uses the original cosine similarities, so we do not repeat the details here. The rationale for std×max⁡\mathrm{std}\times\max and the comparison with a learned scalar gate have also been addressed in the previous response and are not repeated here.

AuthorsOpenReview ↗

Response to Reviewer MCmC (Additional Weaknesses)

In addition to the questions above, several points raised in the Critical Weaknesses are not covered by the question list. We address them separately below for completeness.

On BernBypass and StrasBypass (W3). We acknowledge that both subsets correspond to gastric bypass surgery, and we no longer describe them as entirely unseen procedures. As also noted in SPA (Yuan et al., 2025) and the MultiBypass140 paper (Lavanchy et al., 2024), the two subsets differ substantially in phase definitions and surgical protocols, creating a notable domain gap between the Bern and Strasbourg centers. We therefore treat them as two distinct institutions/domains rather than two copies of the same procedure. The revised manuscript distinguishes cross-procedure generalization (Cholec80, AutoLaparo) from cross-institution generalization (BernBypass, StrasBypass) and states clearly that the latter does not constitute evaluation on an entirely unseen procedure type.

On the comparison with HyperVLP and the meaning of “≤5-shot” (W4). We clarify the positioning of Table 2. Table 2 is not intended to claim overall superiority over all baselines; its purpose is to show that, under a stricter annotation budget (a single demonstration video, ≤5% annotation), ICL-Surg still achieves the best result on some datasets while remaining competitive on the others. We acknowledge that ICL-Surg underperforms HyperVLP on BernBypass (62.77 vs. 72.3) and AutoLaparo (58.72 vs. 60.1), and this comparison is not under an equivalent annotation budget: HyperVLP uses a 10-shot setting, while ICL-Surg uses a single demonstration video (≤5% annotation), as stated in Table 2.

The Shot column in Table 2 follows the same definition as in HyperVLP (Hu et al., 2026), from which the baseline numbers are taken: a shot denotes the fraction of annotated data relative to the total dataset size. The “10” for the baselines therefore corresponds to 10% annotated data, and our “1 video (≤5)” means that the single demonstration video used as context accounts for no more than 5% of the annotated windows of the dataset. This is distinct from the K-shot used during training (K=5), which refers to the number of support windows sampled per class within each episode. At inference, the context is one complete demonstration video, and for each class present in the demonstration the prototype is the average over all annotated windows of that class; this is consistent with the ≤5 annotation budget in Table 2, because the entire demonstration is itself below 5% of the dataset, and the per-class window count is not capped at 5.

Reviewer_c6VyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies few-shot surgical phase recognition: given a single annotated demonstration video, predict the phase for each window of a query video. ICL-Surg extracts features with a frozen PeskaVLP, models within-window temporal structure with a GRU in a local branch, and obtains a video-level representation with cumulative max pooling in a global branch. The two cosine-similarity streams are adaptively weighted by Dynamic Confidence-aware Fusion (DCF) according to a confidence score. Workflow-guided Logic Refinement (WLR) then applies Viterbi decoding, with transition and positional priors obtained by linearly interpolating statistics from the demonstration video and a label pool. Experiments use leave-one-dataset-out evaluation on Cholec80, AutoLaparo, BernBypass, and StrasBypass, reporting frame-level accuracies of 68.14/58.72/62.77/70.84, outperforming D2ST-Adapter, TEAM, FSL, and other few-shot methods; ablations show that WLR contributes +8.65 to +12.85.

Critical strengths
  • Method level: lightweight, deployable, and consistently effective across four datasets in the one-shot setting. Frozen encoder + GRU + prototypes + Viterbi, with only about 3.5M trainable parameters and no gradient updates at inference. For clinical use, "adapt to a new procedure from one demonstration video" is an appealing setting. The gains over D2ST-Adapter/TEAM/FSL in Table 1 are substantial, indicating that "visual matching + workflow priors" is indeed effective.
  • Method level: the dual-source prior in WLR is a sensible design, and the ablation honestly shows that it is the largest contributor. Linearly interpolating the local transition statistics from the demonstration with the empirical statistics from a label pool alleviates the sparsity of transitions in a single video. In Table 3, WLR contributes +8.65 (Cholec80) to +12.85 (StrasBypass), and this finding is itself useful for future few-shot work.
Critical weaknesses
  • Problem/protocol level: the boundary of "few-shot" needs clarification. The paper states that MempM_{emp} and the empirical positional priors are computed "from the entire training set" and "from a label pool that aggregates phase label sequences from all training videos." Under leave-one-dataset-out evaluation, if "training set" here means the target dataset's own training videos, then the model uses the full phase annotations of the target procedure to build transition/positional priors, which is no longer strictly one-shot/few-shot. If it means the other six datasets, then the alignment of phase label spaces across datasets (Cholec80 7 classes, BernBypass 12 classes, AutoLaparo 7 classes) also needs to be explained. This is the point that most needs clarification, because it directly determines how readers interpret the paper's positioning and the gains in Table 3.
  • Method/wording level: the causality of the global branch and the wording in Table 2 could be more careful. Eq. 3 defines gt=max⁡(gt−1,ht)g_t=\max(g_{t-1},h_t), giving a final gTvg_{T_v} at t=Tvt=T_v, but Sec. 3.3 uses gjg_j for each query window jj. If the final gTvg_{T_v} is actually used, then all windows share future-frame information, and the method is inherently offline and not directly comparable to online phase recognition. This may also explain why "Global only" already reaches 54.50 on Cholec80. I would suggest stating this explicitly and adding a causal version that uses only past windows. In addition, in Table 2, BernBypass is 62.77 vs 72.3 for HyperVLP, a 9.5-point gap; describing this as "remains competitive" is somewhat optimistic. AutoLaparo (58.72 vs 60.1) can reasonably be called close, but the BernBypass gap should be discussed honestly, along with whether the shot counts are comparable.
Questions for authors
  • Could you clarify, for each leave-one-dataset-out run, which videos are used to compute MempM_{emp} and the empirical positional priors? Are any labels from the target dataset used? If so, please add a version that uses priors from the other six datasets only; if not, please explain how phase labels are mapped across datasets.
  • Is gjg_j the cumulative max up to window jj, or the final gTvg_{T_v}? If it is the latter, please state that this is an offline setting and add a causal version that uses only past windows; this would make the comparison with online methods clearer.
  • The confidence in Eq. 5 depends on IcI_c (the support set of class cc), but αj\alpha_j in Eq. 6 has no cc subscript, and Eq. 7 treats it as a per-sample scalar. Could you clarify whether α\alpha is per-class or per-sample, and provide a DCF ablation (max-only, entropy, learned scalar)?
  • Ot(yt)O_t(y_t) in Eq. 15 comes from the DCF max cosine and may be negative; how is log⁡O\log O defined? Is a softmax/temperature normalization applied? Could you provide the implementation details?
  • Could you add two key baselines: (a) Viterbi using only the empirical prior, without visual observations; and (b) directly fine-tuning a supervised phase recognizer on the demonstration video? The first would show whether WLR's gain mainly comes from "the prior itself being strong," and the second is the most natural control in the one-shot setting.
  • Could you compare with SPA (Yuan et al., 2025) and Meta-SurDiff? Both are surgical few-shot/meta methods and are currently only mentioned in related work; adding empirical comparisons would clarify the positioning.
  • In Tables 3/4, ICL-Surg is trained on the training set of StrasBypass/BernBypass and then tested, whereas SurgVLP is a pretrained model. Is this comparison fair, given that it differs from the leave-one-out protocol of the main experiments?
  • Table 1 does not report variance over five seeds; could you add it and perform a significance test for gaps such as 68.14 vs 49.61? Also, is the bold in Table 2 correct (ICL-Surg is not the best on BernBypass)?
  • In the few-shot setting, if a query video contains a phase absent from the demonstration video, the model can never predict that class. How large is the effect of this closed-set limitation on Cholec80/BernBypass? Could you report per-class recall?
3 comments
AuthorsOpenReview ↗

Response to Reviewer c6Vy (Q1–Q4)

We thank the reviewer for the detailed and insightful comments.We respond to each comment below:

Response to Q1/W1:

In the leave-one-dataset-out evaluation, no video or label from the target dataset is used to estimate �0��0� or the empirical positional priors. For each test dataset D, the priors are computed exclusively from the video labels of the other six datasets; the target dataset D does not participate in training or prior construction. The setting therefore remains strictly few-shot: phase annotations of the target procedure are unseen before testing.Regarding the alignment of phase label spaces across datasets, we do not rely on semantic label alignment. The phase labels in each dataset follow the official phase definitions and are numbered in workflow order so the labels are ordinal indices (0, 1, 2, …) rather than semantic phase IDs. M{emp} and the positional priors are therefore estimated directly in ordinal space. The datasets have different numbers of classes (e.g., Cholec80 7, BernBypass 12, AutoLaparo 7) but this does not affect ordinal statistics: the priors capture coarse-grained workflow structure shared across procedures including ordered progression, the relative position of the k-th phase, and permissible ordinal transitions. They do not involve a mapping between semantic labels. This is why WLR applies to unseen procedures.

Response to Q2/W2:

Eq. (3) defines gt=max⁡(gt−1,ht)g_t = \max(g_{t-1}, h_t), where hth_t is the GRU output of the tt-th window. The global feature is updated after each window. gTvg_{T_v} is simply the value of this recursion at t=Tvt = T_v, obtained after processing all windows, so it is not a final global feature shared by all query windows. The gjg_j used for query window jj in Sec. 3.3 is exactly the cumulative value of Eq. (3) at t=jt = j, using only current and past windows and no future information.

Response to Q3:

The DCF weight αj\alpha_j is by design a per-sample scalar. The structure of Eq. (5) is correct, and the ambiguity comes from the description of Ic\mathcal{I}_c. The original text describes Ic\mathcal{I}_c as "the set of support sample indices for the cc-class", which can mislead readers into thinking that std/max are computed over the support samples of a single class, yielding a per-class confidence. In the implementation, ii indexes all class prototypes. For query window jj, we compute its similarities to all class prototypes, apply the std/max statistics in Eq. (5) over this similarity vector to obtain per-sample confidences, obtain the scalar αj\alpha_j via Eq. (6), and fuse via Eq. (7). This is consistent with Eqs. (6)–(7), where αj\alpha_j has no class subscript and is one scalar per window. The revised manuscript will redefine I\mathcal{I} in Eq. (5) as the set of all class prototypes, i.e., I={1,…,C}\mathcal{I}=\{1,\ldots,C\}. Besides,We have completed the DCF ablation with alternative designs and updated the manuscript accordingly. Results (mean ±\pm std over 5 runs) are:

Datasetstd$\times$maxmax-onlyentropylearned scalar
Cholec8059.49 ±\pm 2.3059.14 ±\pm 2.5859.04 ±\pm 2.8955.20 ±\pm 2.11
StrasBypass57.99 ±\pm 1.5957.30 ±\pm 3.6856.76 ±\pm 2.9356.89 ±\pm 3.25

The proposed std$\times$max is best on StrasBypass and comparable to max-only and entropy on Cholec80, while all three per-sample adaptive variants clearly outperform the learned scalar. This confirms that per-sample adaptation is the main source of DCF's gain, and we retain std$\times$max as the default. The revision has added this ablation in Sec. 4.3 and Table 4.

Response to Q4:

Ot(yt)O_t(y_t) is not the raw cosine similarity but the DCF-fused similarity after softmax normalization: the fused similarity is scaled by τ=0.1\tau=0.1 and passed through a softmax, so log⁡Ot\log O_t is well defined. The revision has added this softmax normalization detail before Eq. (15).

AuthorsOpenReview ↗

Response to Reviewer c6Vy (Q5–Q8)

Response to Q5:

We thank the reviewer for these two suggestions. Both baselines have been added to the revision.

(a) Prior-only Viterbi (no visual observations). We set the observation term OtO_t in Eq. (14) to a uniform distribution and decode using only the fused transition and positional priors. Results are:

DatasetICL-Surg (Full)Prior-only Viterbi
Cholec8068.14 ±\pm 1.5340.77
StrasBypass70.84 ±\pm 1.3042.04

Prior-only Viterbi is far below ICL-Surg on both datasets (−27.37 and −28.80), confirming that WLR's gain comes from jointly combining visual evidence with workflow priors rather than from the prior alone.

(b) Fine-tuning a supervised phase recognizer on the demonstration video. We fine-tune the GRU and a classification head on the demonstration video, initializing the GRU from the meta-trained ICL-Surg checkpoint. Results (mean ±\pm std over 5 runs) are:

DatasetICL-Surg (Full)Finetune (ICL-Surg init)
Cholec8068.14 ±\pm 1.5353.14 ±\pm 2.31
AutoLaparo58.72 ±\pm 2.0460.04 ±\pm 0.52
BernBypass62.77 ±\pm 2.3348.57 ±\pm 2.47
StrasBypass70.84 ±\pm 1.3044.27 ±\pm 0.40

Even with the same meta-trained initialization, fine-tuning on a single demonstration underperforms ICL-Surg on three of four datasets and remains highly unstable (44.27–60.04) despite converging to near-zero training loss on the demonstration. This indicates that ICL-Surg's gain does not come from the meta-trained representation alone: per-video fine-tuning overfits the single demonstration, whereas prototype matching with DCF and WLR remains stable across all datasets (58.72–70.84).

Response to Q6:

We thank the reviewer for the suggestion to compare with these methods. However, they differ too substantially in setting from ours for a meaningful direct numerical comparison. SPA (Yuan et al., MICCAI 2025) is a multimodal few-shot adaptation framework but its shots are image-level annotations whereas in ICL-Surg a shot is a video-level demonstration. SPA defines phases in natural language text, aligns visual-text embeddings, and relies on a task graph as an explicit temporal prior. Ours is a purely visual in-context few-shot method: prototypes are built from the visual windows of a demonstration video with no text phase definitions and no external task graph. The two therefore differ in shot granularity, input modality and prior dependency so accuracy gaps cannot be attributed to the method itself. Meta-SurDiff is a fully supervised online phase recognition method.Its meta-learning is a re-weighting objective to address phase imbalance, not a few-shot adaptation mechanism and training at test time still requires fully supervised target-domain data. The few-shot setting is orthogonal and direct comparison would not reflect ICL matching capability.

Response to Q7:

We thank the reviewer for raising this fairness concern. We first clarify that we understand the reviewer to refer to Table 2 (Table 3 is the ablation study and Table 4 reports dataset statistics).

Tables 1 and 2 serve different purposes. Table 1 compares all methods under an identical setting and is the main fair comparison. Table 2 is not intended to claim overall SOTA. As stated in the paper, ICL-Surg uses "a stricter and more practical setting than all baselines" and "this comparison is therefore conservative in favor of the baselines." Under this setting, ICL-Surg achieves the highest accuracy on Cholec80 and StrasBypass rather than “outperforming all baselines.”

Response to Q8:

ICL-Surg results in Table 1 are averaged over 5 independent runs; baseline results were obtained by re-training and re-testing each baseline under our own experimental setting using exactly the same evaluation protocol as ICL-Surg. Due to computational cost, each baseline was run once rather than repeated five times.The issues related to Table 2 have been addressed in our previous response. The numbers in Table 2 are correct.

AuthorsOpenReview ↗

Response to Reviewer c6Vy (Q9)

Response to Q9:

We acknowledge that under the ICL protocol, if a phase does not appear in the demonstration video, the model cannot build a prototype for that phase and therefore cannot predict that class. We exclude these windows at test time because under the ICL protocol, the model cannot predict classes absent from the demonstration video and including these windows in evaluation would not reflect the model's performance.We have added a per-phase analysis in Appendix C, including per-phase precision, recall, and F1 for all four test datasets as well as a quantitative analysis of the closed-set limitation.

The key results are as follows. On Cholec80 and AutoLaparo, the demonstration covers all phase classes so no windows are excluded and the closed-set limitation does not apply. On BernBypass, phases 0 and 2 are absent from the demonstration, accounting for 9.86% of the test windows; excluding them gives an overall accuracy of 59.87% ± 1.41% while including them drops it to 53.97% ± 1.27%, a reduction of 5.90 percentage points. On StrasBypass, only phase 11 is absent (0.10% of the test set) and including it changes the overall accuracy by only 0.07 percentage points.

The per-class recall in Appendix C further shows that the low-recall phases (e.g., phases 4, 5, and 9 on StrasBypass, and phases 4, 5, 6, and 8 on BernBypass) are those visually similar to neighboring phases rather than being caused by the closed-set limitation. Overall, the closed-set limitation is negligible on Cholec80, AutoLaparo and StrasBypass, and is non-trivial only on BernBypass where the demonstration does not cover all phase classes. We have stated this explicitly in Appendix C and in the limitations discussion.

Reviewer_7thiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper present a Workflow-guided Logic Refinement (WLR) module that applies Viterbi decoding with dual-source priors.

Critical strengths

The combination of visual similarity matching and structured temporal refinement is a sensible design.

Critical weaknesses
  • The main contribution appears to be the integration of existing components (GRU, Cumulative max pooling) into a few-shot surgical phase recognition framework. Although the in-context adaptation module is a newly proposed module, it is not immediately clear what new ICL capability the framework introduces.

  • The paper claims to recognize surgical phases in procedures unseen during training. However, the WLR module estimates an empirical transition matrix and positional priors from the entire training dataset. This introduces a question: "How can empirical transition probabilities from different surgical procedures be applied to phase classes that were not observed during training?" For example, Cholec80 contains seven phases associated with cholecystectomy, while AutoLaparo has seven phases associated with laparoscopic hysterectomy. These phases have different semantic definitions and potentially different procedural structures.

  • The paper uses a leave-one-dataset-out evaluation protocol, training on six datasets and testing on the remaining dataset. However, BernBypass and StrasBypass are subsets of the same MultiBypass140 benchmark, representing the same general surgical procedure at two institutions. Therefore, when BernBypass is the held-out test dataset, StrasBypass remains in the training collection, and vice versa.

Questions for authors
  • How are empirical workflow priors transferred between different surgical procedures with different phase vocabularies?

  • What exactly constitutes one shot in ICL-Surg? Does the method use five sampled annotated windows per class, or all annotated windows from one complete demonstration video?

  • Does the reported generalization reflect truly unseen procedures or merely new institutions?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 7thi

We sincerely thank the reviewer for the constructive comments.Our responses are provided below:

Response to W1:

We clarify that we do not claim GRU or cumulative max pooling as novel components; they serve only as lightweight temporal encoders. The new ICL capability introduced by this work is concentrated in two novel modules: (i) DCF (Dynamic Confidence-aware Fusion), which adaptively fuses local and global similarities based on per-query confidence under the demonstration condition without additional supervision, enabling ICL to exploit the two complementary representations more reliably; and (ii) WLR (Workflow-guided Logic Refinement), which couples the ICL matching outputs with surgical workflow logic using demonstration and ordinal empirical priors for Viterbi-based sequence refinement so that predictions conform to phase ordering. This capability is absent in existing ICL/prototypical networks. The contribution is therefore not a component stack but an ICL mechanism tailored to surgical workflow structure.

Response to Q1/W2:

We clarify that the phase labels in each dataset follow the official phase definitions and are numbered in workflow order so all labels are ordinal indices (0, 1, 2, …) rather than semantic phase IDs. The empirical term in WLR is therefore estimated directly in ordinal space: it aggregates transition and positional statistics by label index across the training datasets and captures coarse-grained workflow structure shared across procedures such as ordered progression, the relative position of the k-th phase, and permissible ordinal transitions. No semantic alignment between phase vocabularies is performed or needed. In each leave-one-dataset-out run, the empirical priors are estimated only from the six datasets other than the held-out target dataset; no video or label from the target dataset contributes to the priors.

Response to Q2:

In ICL-Surg, one “shot” refers to one annotated temporal window (L = 16 frames), not a complete video. During training, each episode samples 5 support windows per class from the demonstration video and 5 query windows per class from the query video. During testing, the context is one complete demonstration video; for each class present in the demonstration, the prototype is the average over all annotated windows of that class in the video. The Shot column in Table 2 denotes the annotation budget: the other baselines use a 10-shot setting while “1 video (≤5)” means that only one demonstration video is used as context, whose annotated data accounts for no more than 5% of the dataset, thus requiring less annotation than the 10-shot baselines.

Response to Q3/W3:

We acknowledge that BernBypass and StrasBypass both come from MultiBypass140 and both correspond to gastric bypass surgery. Under the leave-one-dataset-out protocol, when one is held out, the other remains in the training set. We should therefore not claim that all test sets correspond to procedure types unseen during training.We note that prior work has explicitly reported that, despite both being gastric bypass surgeries, the two subsets differ substantially in phase definitions and procedural protocols creating a significant domain gap.SPA(Yuan et al., MICCAI 2025)states that “differences in phase definitions and procedural protocols create a significant domain gap, requiring adaptation of surgical foundation models”. We therefore treat BBP and SBP as two distinct domains/institutions rather than two copies of the same procedure. For test sets such as Cholec80 and AutoLaparo, the training collection contains no surgery of the same type, so these evaluations do reflect unseen procedures; for BBP and SBP, the evaluation reflects cross-institution generalization rather than cross-procedure generalization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.