acceptodds
Withdrawn from ICLR 2027

SurgHiCo: Hierarchical Composition of Long-Horizon Surgical Video–Language Representations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Surgical video-language pretraining uses narrated videos and their hierarchical captions, from individual maneuvers to stretches of a procedure. However, clip-based baselines encode segments with a fixed frame budget or aggregate local features by mean pooling, providing limited modeling of how local events form longer sequences. Under matched controls, neither incorporating hierarchical captions nor enlarging the temporal receptive field substantially improves the baseline, suggesting that the bottleneck lies in how longer-range representations are formed rather than in supervision or input. Motivated by this finding, we propose SurgHiCo, a hierarchical video-language pretraining framework that organizes representation learning by temporal scale, following the structure of surgery itself: maneuvers make up steps, and steps make up phases. Each level is composed from the level below by a lightweight composer and aligned with text at its own span. SurgHiCo achieves state-of-the-art zero-shot surgical phase recognition across five datasets and the strongest linear-probing performance at every labeled-video fraction on all of them. Notably, it improves average zero-shot accuracy by 6.4 points over the strongest prior model and full-shot linear-probing macro-F1 by 12.2 points.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
surgical video understanding, video-language pretraining, hierarchical representation learning, temporal composition, zero-shot surgical phase recognition

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NBPhOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SurgHiCo, a surgical video–language pretraining framework that builds local, intermediate, and long-horizon representations in three stages. After training a local encoder, it progressively freezes earlier visual modules and trains Transformer composers using captions at corresponding temporal scales. Long-horizon captions are constructed by summarizing and concatenating consecutive intermediate captions from SurgLaVi-β. The framework is evaluated through zero-shot phase recognition and linear probing on five datasets. I have the following comments.

Critical strengths

1、A practical approach to extending frozen surgical video representations. The staged design allows the higher-level modules to train on cached features without repeatedly decoding videos or updating the backbone. Evaluation across five datasets, together with caption- and input-matched mean-pooling controls, provides useful evidence that the complete approach improves over simple aggregation.

Critical weaknesses

1、The experiments do not establish the contribution of the proposed hierarchy. HierVL (CVPR 2023) already learns representations at different temporal scales using self-attention aggregation and language supervision. SurgHiCo introduces a three-level construction and progressively frozen training, but the experiments do not isolate the value of these choices. The main control in Table 2 uses parameter-free mean pooling, while SurgHiCo adds trainable composers, staged optimization, and text adapters. There is no comparison with a non-hierarchical Transformer aggregator of comparable capacity, or an ablation of progressive freezing. The reported improvement therefore leaves the paper’s main methodological claim unresolved.

2、The baseline comparisons leave uncertainty about the reported advantage. Table 1 compares SurgHiCo’s 256-frame input with published results using 16-frame inputs. Section 4.2 acknowledges this difference, but the headline comparison still relies on these unequal settings. The controlled experiments also need a stronger baseline check: the reproduced flat F/M/C model averages 39.47%, compared with 44.04% for the quoted SurgCLIP-β result. The discrepancy is particularly large on StrasBypass, at 16.58% versus 31.24%. The paper does not explain this gap. Until the reproduction is reconciled and the strongest baseline is evaluated under comparable input conditions, the magnitude of the improvement remains difficult to assess.

Questions for authors

1、How does SurgHiCo compare with a single Transformer that aggregates the same 16 local features, using the same caption supervision and a comparable parameter budget?

2、What happens when the hierarchy is trained without progressively freezing earlier visual modules? Is there evidence that freezing contributes to the results independently of the added composers?

3、What accounts for the gap between the reproduced flat F/M/C baseline and the quoted SurgCLIP-β results, particularly on StrasBypass? Can the released SurgCLIP-β checkpoint be evaluated with the same prompts, test videos, and long-context input used for SurgHiCo?

4 comments
AuthorsOpenReview ↗

Response to Reviewer NBPh: Novelty Relative to HierVL

Clarification of Novelty Relative to HierVL

We thank the reviewer for highlighting HierVL as relevant prior work. Although both methods explore hierarchical video–language learning, their research motivations differ. HierVL aims to incorporate long-term activity intent through video-level summaries to enrich local action and global video representations. In contrast, SurgHiCo addresses the limitations of clip-based VLP in representing long-horizon surgical workflows. Surgical procedures exhibit an inherent temporal hierarchy, where individual maneuvers form procedural steps, and steps collectively constitute longer surgical processes. Understanding these workflows requires representations that capture temporal relationships beyond isolated clips.

A related challenge in multi-scale video–language learning is that fine-grained visual details and abstract procedural semantics impose different representational demands. Jointly updating a shared visual encoder with supervision across temporal scales may therefore introduce cross-scale optimization interference.

HierVL addresses the risk of forgetting local representations through alternating training between short- and long-term objectives. SurgHiCo takes a different approach: it progressively freezes previously learned visual modules and employs lightweight composers to explicitly model and combine lower-level visual features into longer-range procedural representations, without repeatedly updating preceding visual modules.

Our additional controlled experiments support this design choice. Under matched trainable parameter budgets, SurgHiCo outperforms flat Transformer aggregation baselines. Furthermore, our freezing ablation shows that preserving the learned Intermediate Composer during long-horizon training achieves higher average accuracy than allowing it to be updated (50.41% vs. 43.84%).

This framework also offers two practical advantages. First, it retains local, intermediate, and long-horizon representations for independent or combined downstream use. Our linear-probing experiments show that combining all three scales outperforms the strongest individual scale across three datasets. Second, once local features are cached, higher-level composers require only a few minutes of additional training, without repeatedly processing raw videos or updating the expensive visual backbone.

The key contribution of SurgHiCo lies in its progressive hierarchical composition and stage-wise freezing strategy, enabling efficient construction, preservation, and reuse of multi-scale video–language representations for long-horizon surgical workflow understanding.

AuthorsOpenReview ↗

Response to Reviewer NBPh: Capacity-Matched Flat Transformer

Comparison with a Capacity-Matched Flat Transformer

To investigate whether the improvements stem from hierarchical modeling rather than increased model capacity, we conducted additional experiments with a capacity-matched flat Transformer.

Specifically, we replace the hierarchical composers (2L Mid + 2L Workflow) with a single 4-layer Transformer that directly aggregates local visual features through global self-attention, without intermediate hierarchical representations. We evaluate two training strategies: Flat-Joint, which jointly optimizes the model using Mid and Workflow supervision, and Flat-Staged, which sequentially applies Mid and Workflow supervision. All methods share the same frozen Stage1 features, caption supervision sources, 16 local features at inference, and evaluation protocol. The trainable parameter counts are nearly identical (28.594M vs. 28.590M; 0.016% difference).

Table R1. Comparison with capacity-matched flat Transformers (video-averaged accuracy, %, seed 42).

ModelCholec80HeiCholeBernStrasAutoLaparoAvg
Flat-Joint (4L)52.7555.0833.9321.3055.3143.67
Flat-Staged (4L)49.8649.1438.0729.0851.8743.60
SurgHiCo (2L+2L)60.2863.0439.0230.1459.5550.41

As shown in Table R1, SurgHiCo outperforms both flat baselines across all five datasets, improving average accuracy by 6.7 pp over Flat-Joint and 6.8 pp over Flat-Staged. Notably, the two flat variants achieve similar average accuracy despite their different training strategies.

These results show that simply increasing Transformer capacity and applying the same multi-scale language supervision, either jointly or sequentially, does not reproduce SurgHiCo's performance. This provides empirical support for the effectiveness of our progressive hierarchical learning framework beyond model capacity alone.

AuthorsOpenReview ↗

Response to Reviewer NBPh: Progressive Freezing and SurgCLIP Reproduction

Effect of Progressive Freezing

To isolate the contribution of progressive freezing from the hierarchical architecture, we conducted an additional ablation in which the Mid composer remains trainable during Workflow-level training.

Both variants retain the identical hierarchical architecture (2L Mid + 2L Workflow), Stage2 initialization, caption supervision, and training schedule. The only difference is whether the Mid composer is frozen during Stage3, allowing us to evaluate the effect of freezing independently of the additional composers.

Table R2. Ablation of progressive freezing (video-averaged accuracy, %).

ModelCholec80HeiCholeBernStrasAutoLaparoAvg
SurgHiCo (Unfrozen)55.8953.5435.2124.0650.4943.84
SurgHiCo (Frozen)60.2863.0439.0230.1459.5550.41

As shown in Table R2, progressive freezing improves average accuracy from 43.84% to 50.41% (+6.57 pp), with improvements across all five datasets. These results provide direct evidence that freezing contributes to the performance of SurgHiCo beyond the addition of hierarchical composers.

Importantly, our primary motivation remains progressive representation learning across temporal scales. Freezing serves as a complementary training strategy that preserves previously learned intermediate representations while enabling efficient long-horizon training through cached features. We will clarify this distinction in the revised manuscript.

SurgCLIP-β Baseline Reproduction and Fair Comparison

We acknowledge that the discrepancy in baseline reproduction and the difference in temporal input lengths may affect the assessment of SurgHiCo's performance gains. To address both issues, we conducted additional experiments using the officially released SurgCLIP-β checkpoint and matched temporal inputs.

(1) SurgCLIP-β Baseline Reproduction.

During our research, we made multiple attempts to reproduce SurgCLIP-β but were unable to fully recover the performance reported in the original paper. The reproduced results presented in our submission represent the best performance obtained across our reproduction attempts. Nevertheless, a gap remains, and we have not identified its definitive cause.

To avoid relying on our independently reproduced checkpoint, we directly evaluated the officially released SurgCLIP-β checkpoint. Under our downstream evaluation protocol, the released checkpoint achieved 44.12% average accuracy across five datasets, closely matching the 44.04% reported in the original paper.

Notably, on StrasBypass, the released checkpoint achieves 31.01%, closely matching the reported 31.24%, whereas our independently reproduced flat baseline achieves 16.58%. Although the cause of the reproduction gap remains unclear, this confirms that the released checkpoint closely recovers the reported performance under our evaluation protocol.

We further used the released checkpoint as SurgHiCo's frozen Stage1 encoder, while keeping the Stage2 and Stage3 progressive hierarchical training procedure unchanged. This configuration achieved 53.86% average accuracy, compared with 50.41% using our independently trained Stage1 encoder.

These results demonstrate that SurgHiCo's effectiveness does not depend on our independently reproduced Stage1 checkpoint and that the framework can effectively extend the officially released pretrained visual representations.

AuthorsOpenReview ↗

Response to Reviewer NBPh: Fair Comparison under Matched Long-Context Input

Fair Comparison under Matched Long-Context Input

To further ensure a fair comparison, we evaluated the official SurgCLIP-β checkpoint under the same downstream evaluation protocol, using the same prompts and test videos. We additionally constructed a long-context Mean Pooling baseline to distinguish the benefits of increased temporal coverage from those of hierarchical composition.

Specifically, we extract 16 local visual tokens from a centered 256-second temporal window, with each local clip containing 16 sampled frames (256 frames in total). We compare two aggregation strategies using the official SurgCLIP-β checkpoint:

  • SurgCLIP-β + Mean Pooling: Averages the 16 local visual tokens into a single representation.
  • SurgHiCo: Applies the proposed hierarchical composition to the same 16 local visual tokens.

For reference, we also report the official checkpoint's original 16-frame evaluation result.

Table R3. Zero-shot comparison using the official SurgCLIP-β checkpoint under different temporal contexts (video-averaged accuracy, %).

DatasetOfficial (16F)Mean (256F)SurgHiCo (256F)
Cholec8057.4664.7861.70
HeiChole57.7257.2760.50
Bern18.1216.9844.50
Stras31.0132.1243.20
AutoLaparo56.2958.4559.40
Average44.1245.9253.86

As shown in Table R3, simply extending the temporal input and applying mean pooling improves the average accuracy from 44.12% to 45.92%. However, under the same 256-frame input context, SurgHiCo achieves 53.86%, outperforming Long-context Mean Pooling by 7.94 percentage points.

Notably, SurgHiCo achieves higher accuracy on four out of five datasets, with particularly substantial improvements on BernBypass and StrasBypass. Although Mean Pooling performs better on Cholec80, the overall results demonstrate that the improvements cannot be attributed simply to increased temporal coverage.

Together with the official checkpoint evaluation in our previous response, these experiments demonstrate that SurgHiCo can effectively extend the released SurgCLIP-β representations and achieve substantial improvements under matched long-context evaluation.

We sincerely thank the reviewer for these constructive suggestions, which have helped us strengthen the experimental validation and clarify the contributions of SurgHiCo. We are currently preparing the revised manuscript and incorporating these additional controlled comparisons and ablation results into the corresponding experimental sections.

Reviewer_hKywOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SurgHiCo for long-horizon surgical video understanding.

Critical strengths
  • The proposed method constructs representations at three temporal scales through lightweight Transformer composers.

  • Competitive zero-shot surgical phase recognition performance across five datasets are reported.

Critical weaknesses
  • The central claim is that hierarchical composition outperforms flat representation learning. However, the main controlled long-context baseline uses parameter-free mean pooling to aggregate clip representations. This comparison is not fully convincing because SurgHiCo introduces additional trainable Transformer composers, while the baseline does not learn how to aggregate temporal information.

  • Analysis of whether the framework learns meaningful temporal relationships is limited.

  • The long-horizon supervision is constructed using Gemini-generated rewrites of existing intermediate-level captions. Although the authors provide details about caption generation, the quality and semantic fidelity of these synthetic descriptions are not quantitatively evaluated.

Questions for authors
  • How to compare the proposed synthetic captions with original coarse-level annotations, direct concatenation of intermediate captions, and alternative caption-generation strategies?

  • How to conduct temporal-order perturbation experiments by shuffling the order of local or intermediate features, reversing surgical step sequences, or removing positional embeddings?

Reviewer_MEv6OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SurgHiCo is a hierarchical video–language pretraining framework for surgical video. The authors first show how a flat clip model gains little from extra longer-span captions or from a wider input with mean pooling. Instead, they argue the bottleneck is how long-range representations are formed. Their method trains in three stages. The text side is adapted with LoRA. Their validation includes the following: Zero-shot trials, linear probing as well as an ablation study.

Critical strengths

The authors perform a thorough validation of their method and compare it to other while demonstrating a consistent, sizeable gain with a cheap design in accuracy (almost 10%).

Critical weaknesses

I think the biggest weakness of the paper is the baselines compared against. First, mean-pooling is the only long-context baseline (not learned vs learned) that they compare against, the gain vs the baseline therefore cannot be attributed to hierarchy rather than to having any learned temporal aggregator, a flat encoder with the same learned composer is the missing control. Also they compare a model given 256 frames against other models given only 16 frames (acknowledged for table 1, but also effects table 2) . Also some design choices, e.g. the freezing of layers vs. end-to-end training, window size, leaving out layers, should also have been ablated.

Questions for authors

The way I understand it, the flat F/M/C model is described as reproducing SurgCLIPs training, yet performs vastly differently. What differs between the two? For linear probing, were the baselines given the same 256 centered context, e.g. with mean-pooled or concatenated features? On what data was the stage 3 batch size chosen? How sensitive are results to the window size and to dropping the intermediate stage?

5 comments
AuthorsOpenReview ↗

Response to Reviewer MEv6: SurgCLIP Reproduction

Response to Reviewer MEv6

Q1. Reproduction Discrepancy with SurgCLIP

Reviewer Concern:

The flat F/M/C model is described as reproducing SurgCLIP's training, yet performs vastly differently. What differs between the two?

Response:

During our research, we strictly followed the training procedure described in the original SurgCLIP paper to independently reproduce its flat F/M/C multi-scale training strategy. However, we observed different experimental results from those reported in the original paper.

Specifically, our reproduced Flat F/M/C achieves an average accuracy of 39.47% across five datasets, compared with 38.09% for the Fine-only F baseline, yielding only a modest improvement of 1.38 percentage points. This differs from the stronger benefit of multi-scale supervision reported in the original SurgCLIP paper. This observation was also one of our motivations for exploring hierarchical representation learning and developing SurgHiCo.

Furthermore, our reproduced Flat F/M/C achieves 39.47% average accuracy, substantially below the 44.04% reported for SurgCLIP-β in the original paper. The discrepancy is particularly pronounced on StrasBypass, where our reproduction achieves 16.58%, compared with 31.24% in the original paper.

We devoted considerable time and effort to repeated reproduction and debugging. Despite these attempts, we were unable to fully recover the reported performance, and the results presented in our paper represent the best performance obtained across our reproduction attempts. We have not yet identified the definitive cause of this discrepancy.

To avoid letting this reproduction issue affect the evaluation of SurgHiCo itself, we further conducted experiments using the officially released SurgCLIP-β checkpoint. Since subsequent hierarchical learning in SurgHiCo builds upon the visual representations learned in Stage 1, we also hypothesized that the quality of the reproduced Stage-1 representations might limit the final performance of SurgHiCo.

First, we directly evaluated the official SurgCLIP-β checkpoint under the same downstream evaluation protocol. It achieves 44.12% average accuracy across the five datasets, closely matching the 44.04% reported in the original paper. On StrasBypass, it achieves 31.01%, also close to the reported 31.24%.

We then replaced SurgHiCo's Stage-1 encoder with the official SurgCLIP-β checkpoint while keeping the Stage-2 and Stage-3 hierarchical training procedures unchanged.

This configuration achieves 53.86% average accuracy, further improving upon the 50.41% obtained with our independently trained Stage-1 encoder.

To determine whether this improvement simply comes from increased temporal coverage, we also constructed a Long-context Mean Pooling baseline using the official SurgCLIP-β checkpoint under the same 256-frame input context.

Table R1. Zero-shot comparison using the official SurgCLIP-β checkpoint (video-averaged accuracy, %).

DatasetOfficial (16F)Mean (256F)SurgHiCo (256F)
Cholec8057.4664.7861.70
HeiChole57.7257.2760.50
Bern18.1216.9844.50
Stras31.0132.1243.20
AutoLaparo56.2958.4559.40
Average44.1245.9253.86

As shown in Table R1, simply extending the temporal input and applying mean pooling improves the average accuracy of the official SurgCLIP-β from 44.12% to 45.92%.

In contrast, SurgHiCo achieves 53.86% under the same 256-frame input context, outperforming Long-context Mean Pooling by 7.94 percentage points, with higher accuracy on four out of five datasets.

Although Mean Pooling performs better on Cholec80, SurgHiCo achieves particularly substantial improvements on BernBypass and StrasBypass.

These results demonstrate that:

First, the effectiveness of SurgHiCo does not depend on our independently reproduced Stage-1 checkpoint.

Second, even when using the official SurgCLIP-β pretrained representations under an identical input context, SurgHiCo still achieves substantial improvements, indicating that its advantages cannot be explained simply by increasing the temporal scale.

AuthorsOpenReview ↗

Response to Reviewer MEv6: Fairness of Linear Probing

Response to Reviewer MEv6 (continued)

Q2. Fairness of Linear Probing under the Same 256-Frame Context

Reviewer Concern:

For linear probing, were the baselines given the same 256 centered context, e.g., with mean-pooled or concatenated features?

Response:

To address this question, we compare our proposed SurgHiCo-Fusion with two aggregation baselines constructed from the officially released SurgCLIP-β checkpoint: Mean Pooling and Concatenation. All methods use frozen visual representations, the same downstream data splits, and the full training set.

Feature Construction and Temporal Context

For SurgCLIP-β, we extract 16 local clip features from a centered 256-second temporal window. Each local clip contains 16 sampled frames, resulting in 256 sampled frames in total. Each clip produces a 768-dimensional visual feature. We investigate two aggregation strategies:

  • SurgCLIP-β + Mean Pooling: Averages the 16 local features, resulting in a 768-dimensional representation.
  • SurgCLIP-β + Concatenation: Directly concatenates the 16 local features, resulting in a 12,288-dimensional representation.

In contrast, SurgHiCo-Fusion combines representations learned at three temporal scales:

  • Local: A centered 16-second window.
  • Intermediate: Four consecutive local clips, spanning 64 seconds.
  • Long-horizon: Four consecutive intermediate units, spanning 256 seconds.

Each SurgHiCo branch produces a 256-dimensional post-projection embedding. We L2-normalize each embedding and concatenate the three representations, resulting in a 768-dimensional fused representation.

Linear Probe Configuration

To isolate the quality of the learned representations, all visual encoders and representation modules remain frozen during probing. Each method uses a single linear classification layer with bias, without dropout or additional normalization.

We use the same full-shot probing protocol: 100% of the training videos, cross-entropy loss, 100 training epochs, batch size 1024, SGD with momentum 0.9, learning rate 0.1, no weight decay, and cosine annealing. The selected epoch is determined by validation Macro-F1.

Table R2. Full-shot linear probing under matched temporal coverage (Accuracy / Macro-F1, %).

MethodContextFeature Dim.Linear Params.AutoLaparoBernBypassStrasBypass
SurgCLIP-β + Mean256s768769C69.38 / 42.4479.36 / 30.6078.24 / 55.76
SurgCLIP-β + Concat256s12,28812,289C63.15 / 33.9779.22 / 30.7476.41 / 54.11
SurgHiCo-Fusion256s768769C80.88 / 68.6482.98 / 47.3684.12 / 72.89

All methods use the same centered 256-second temporal context and 256 sampled frames. SurgHiCo-Fusion integrates representations learned at local (16s), intermediate (64s), and long-horizon (256s) scales. C denotes the number of phase classes. Linear Params. counts the trainable linear classifier parameters, including bias, but excludes frozen pretrained modules.

Results and Analysis

As shown in Table R2, SurgHiCo-Fusion consistently outperforms both SurgCLIP-β aggregation baselines across all three datasets in terms of Accuracy and Macro-F1.

Compared with the stronger of the two baselines for each metric, SurgHiCo-Fusion improves Accuracy by 11.50, 3.62, and 5.88 percentage points on AutoLaparo, BernBypass, and StrasBypass, respectively. The corresponding Macro-F1 improvements are 26.20, 16.62, and 17.13 percentage points.

Importantly, SurgHiCo-Fusion and Mean Pooling use the same temporal coverage and an identical 768-dimensional linear classifier, yet SurgHiCo-Fusion achieves substantially better performance. Furthermore, although direct concatenation increases the feature dimensionality and linear classifier parameters by approximately 16 times, it still underperforms SurgHiCo-Fusion.

These results demonstrate that SurgHiCo-Fusion benefits from combining complementary representations learned at different temporal scales, rather than simply increasing temporal coverage, feature dimensionality, or linear classifier capacity.

AuthorsOpenReview ↗

Response to Reviewer MEv6: Batch Size and Window-Size Sensitivity

Response to Reviewer MEv6 (continued)

Q3. Selection of Stage-3 Batch Size

Reviewer Question:

On what data was the Stage-3 batch size chosen?

Response:

The Stage-3 batch size was fixed at 256 across all Stage-3 experiments, based on computational resource availability and training considerations, rather than selected through downstream performance-based hyperparameter search.

During training, we temporarily reduced the batch size due to GPU resource constraints and restored it to 256 once sufficient resources became available. Importantly, the batch size was not selected or tuned based on the performance of any specific downstream dataset or test set.

Q4. Window-Size Sensitivity

Reviewer Question:

How sensitive are results to the window size?

Response:

We conducted additional experiments to investigate the sensitivity to the number of Intermediate representations aggregated by the Long-horizon Composer.

Specifically, we train separate Stage-3 models using 2, 3, and 4 consecutive Intermediate representations, corresponding to nominal temporal contexts of 128, 192, and 256 seconds, respectively. The Stage-1 and Stage-2 representations are kept fixed, while the number of input Intermediate representations is varied.

Table R3. Sensitivity to the number of Intermediate inputs (video-averaged accuracy, %, seed 42).

K (Intermediate Tokens)ContextCholeHeiBernStrasAutoAverage
K=2128s58.5959.8935.1523.2851.2445.63
K=3192s59.2263.5037.4725.5657.0648.56
K=4256s60.2863.0439.0230.1459.5550.41

As shown in Table R3, the average accuracy consistently increases with the temporal context length, from 45.63% with K=2 to 48.56% with K=3 and 50.41% with K=4, yielding an overall improvement of 4.78 percentage points from K=2 to K=4.

Although the optimal window size varies slightly across datasets, K=4 achieves the best overall performance and performs best on four out of five datasets. HeiChole achieves slightly higher accuracy with K=3, suggesting that the optimal temporal context may depend on the characteristics of individual surgical procedures.

These results support our choice of four Intermediate representations as the default configuration, while demonstrating that longer temporal context generally benefits long-horizon representation learning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.