- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies corruption robustness in 4D point cloud understanding, covering action recognition and temporal action segmentation. It introduces an evaluation protocol with five spatial corruptions and three temporal perturbations on MSR-Action3D, NTU-RGBD60, and HOI4D, and evaluates several representative backbones. It also proposes STEADY, which combines an online hierarchical cache of global and local point-tube features with temporal consistency gating. For action segmentation, a dynamics-aware decoder further combines the adapted frame scores with initial-state, transition, and duration priors estimated from training annotations through explicit-duration hidden semi-Markov decoding. Backbone weights remain frozen.
The experiments show modest improvements in average corrupted recognition accuracy and substantially larger improvements in average segmentation edit and F1 scores. However, frame accuracy decreases for some segmentation backbones, as acknowledged in the appendix. Ablations, sensitivity analyses, qualitative examples, and inference-cost measurements provide additional validation.
Critical strengths
- The paper investigates a relevant and comparatively underexplored robustness problem with useful experimental breadth: eight corruption types, three datasets, two tasks, and six distinct backbone architectures across the experiments. Reporting results by corruption type exposes differences that clean performance alone does not reveal. A sufficiently specified and released protocol could provide a useful basis for subsequent work on point cloud video robustness.
- STEADY achieves substantial average segment-level improvements across all four evaluated segmentation backbones without updating neural-network weights. For example, average edit score increases from 60.1 to 73.7 for P4Transformer and from 80.9 to 86.2 for Mamba4D; average [email protected] also improves for both models and the other segmentation backbones. These improvements provide meaningful evidence that the proposed inference procedure can produce more coherent action segments under corruption (Tables 2 and 6).
Critical weaknesses
-
The comparisons do not establish how much HSC and STCG contribute beyond applying the decoder directly to the backbone outputs. A backbone-plus-DAD-only comparison is missing. In Table 3, HSC and STCG together improve spatial and temporal edit scores by 0.41 and 1.12 points, respectively, whereas subsequently adding DAD improves them by another 3.14 and 7.02 points. These results show that the decoder is important within the tested combination, but do not establish the additional benefit of the cache and gate over decoding the original predictions.
-
The paper does not clearly show whether its cache design performs better than existing cache-based methods. Figure 3 tests versions without local features or negative cache entries, but it is unclear whether these versions are adaptations of TDA or Point-Cache. Explaining this connection and comparing results using the same backbone and evaluation settings would help show the benefit of the proposed design.
-
The data and criterion used for hyperparameter selection are unclear, as is the ordering of the recognition test stream. These details matter because the method uses several cache and decoding parameters, and the recognition cache is updated sequentially across test observations. A clearer account of parameter selection and stream construction is needed to assess the evaluation protocol.
-
The paper does not clearly show whether the improvements remain consistent when experiments are repeated with different random noise patterns, point removals, or test-video orders. This matters because the recognition gains are small: 0.5–1.6 percentage points on MSRAction3D-C and 0.6–0.8 points on NTU60-C. Reporting how the gains vary across repeated tests would help establish whether they are reliable.
Questions for authors
1. For action segmentation, how does full STEADY compare with the original backbone plus DAD, without HSC and STCG? Please use the same priors and tuning protocol and report frame accuracy, edit score, and F1.
2. Are the cache versions tested in Figure 3 adaptations of TDA or Point-Cache? Please explain any differences. If not, could you adapt these methods to this task and compare them with your cache using the same backbone and test settings?
3. How did you choose the cache and decoder hyperparameters? Which data and performance measures were used to select them? Please also explain the order in which recognition test videos were processed.
4. How consistent are the recognition improvements across different random corruption samples or different test-video orders? Please report the average gains and how much they vary across repeated evaluations.
5 comments
Thank you for the additional experiments and explanations. The DAD-only comparison addresses my first concern for Mamba4D, and the results across three video orders substantially address my fourth concern. I have two remaining clarification questions:
- To make sure I understand Figure 3 correctly, are the HSC versions without local features or without negative cache entries adaptations of TDA or Point-Cache using the same backbone? If so, could you please identify which version relates to each method and explain any differences in how the caches work?
- When you say the hyperparameters were chosen using clean datasets, does this mean training data, a separate validation set, or the clean test set?
Dear Reviewer ng82,
Thank you for your follow-up questions and for taking the time to review our additional experiments. We are glad that the DAD-only comparison and the evaluation across video orderings addressed your earlier concerns. Below, we clarify how the cache variants in Figure 3 relate to TDA and Point-Cache and specify the data used for hyperparameter selection.
-
Figure 3 presents two component ablations of HSC. The variant without local features resembles TDA. It retrieves positive and negative scores using global features from the frozen point cloud video backbone, then adds the positive scores and subtracts the negative scores from the original logits. TDA uses CLIP image features for retrieval. The variant without negative entries resembles Point-Cache in combining positive scores from global and local retrieval. Its local keys summarize point tubes spanning neighboring frames, while Point-Cache derives its local keys from spatial patches within a single point cloud. In STEADY, HSC uses these tube descriptors to retrieve evidence, and STCG uses feature and prediction consistency across adjacent observations to regulate the strength of the resulting correction. We also provide further details on how HSC constructs, updates and queries its caches in Algorithm 1 of Appendix B.
-
The evaluation protocols we follow for all three datasets do not specify a separate validation split. We therefore held out a subset of the clean training data for hyperparameter selection, using OA for recognition and OE for segmentation. After fixing the hyperparameters, we merged the validation subset back into the training split, trained the backbone on the full training split, and evaluated on the official test split. In addition, the sensitivity analyses in Figures 4 and 7 show that STEADY’s performance is generally stable under moderate changes to the evaluated hyperparameters.
Response to Reviewer ng82
Thank you for your precious suggestions. We address your concerns in as follows.
1. Comparison of full STEADY with backbone plus DAD. On Mamba4D, full STEADY improves OA, OE, and OF by 0.11, 0.93, and 1.82 points over applying DAD directly to the backbone outputs. Both configurations use the same priors and tuning protocol.
Table A. Component ablation on HOI4D-P. OA, OE, and OF are frame accuracy, Edit, and F1 score, averaged over eight corruption types.
2. Differences from TDA and Point-Cache. HSC caches spatio-temporal tube descriptors whose receptive fields span neighboring frames, together with global video features. Unlike TDA’s global-feature caching, this hierarchy retains local geometric and temporal evidence. Unlike Point-Cache’s static spatial patches, each tube incorporates information across frames, and neighboring tubes may share temporal context. HSC and STCG are closely coupled. HSC retrieves temporally contextualized tube evidence, while STCG uses consistency between adjacent observations in feature and prediction space to regulate its contribution.
Structurally, Point-Cache is closer to the HSC-only variant, since both retrieve global and local evidence without STCG. Table A shows that adding STCG to HSC improves OA/OE/OF from 76.79/81.29/81.79 to 76.98/81.54/82.43.
3. Hyperparameter selection and test-video ordering. Hyperparameters were selected on the original clean datasets by maximizing OA for recognition and OE for segmentation, then fixed for all corruption evaluations. Clean recognition videos followed the original dataset order, whereas corrupted recognition videos were processed in random order. We additionally report results across three different test-video orderings in our response in Table B.
4. Sensitivity to test-video ordering. We evaluate each model under three different test-video orderings. STEADY achieves higher mean OA than the corresponding backbone in all evaluated settings.
Table B. Recognition performance under different test-video orderings. Entries report mean OA (%) and standard deviation across three runs. OA averages accuracy over eight corruption types.