- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents BIDETA, a framework for adapting frozen tactile encoders to unfamiliar sensors using a small labeled support set and an unlabeled query batch. The method combines statistical classifiers fitted to two encoder readouts, a graph adjusted using variation in the support examples, and recurrent prediction updates controlled by uncertainty. Using TVL and Sparsh, the authors then evaluate classification and identity ranking tasks across three datasets.
Critical strengths
-
Being able to adapt trained encoders to new tactile hardware with limited labeled data is a useful practical problem. The combination of support memory, spectral adjustment of feature geometry, and graph propagation is justified well; keeping the encoder frozen also makes the approach attractive when retraining is costly.
-
The evaluation across SITR, TacVerse Shape, and TacQuad, with two different pretrained encoders, is a strong point. The classification gains are substantial across most sensors and label settings. I also appreciate the identity ranking results, which provide evidence beyond classification.
-
On SITR with TVL at 10% support, adding the graph and recurrence to the memory improves accuracy from 76.07% to 85.71%. This result really helps understanding their combined contribution.
-
The appendix is very detailed and informative as it includes accuracy and macro F1 by sensor, results with 2, 5, and 10 examples per class, confusion matrices, both ranking metrics, and runtime measurements. Together with the parameter studies and results over three seeds, this makes the experimental evidence much easier to assess.
Critical weaknesses
-
The ablations explain the gains within BIDETA, but the external comparisons use different feature readouts (I see that Table 6 gives BIDETA two readouts while each baseline uses one). I would like to understand how these choices were made and whether giving the baselines access to the same features would change the comparison.
-
I am less convinced by the benefit of the full reliability gate than by spectral normalization: in Table 4, using entropy or disagreement alone gives nearly the same results as the full gate. A comparison with a fixed recurrence weight would help show how much is gained by adjusting the weight for each query.
-
The gains are less consistent with very few support examples. For example, BIDETA with Sparsh falls slightly below the strongest comparison on TacTip and ViTac at 1% support. The paper does suggest that the three examples per class may be unrepresentative, but I’m not sure whether the main difficulty is already present in the support memory or is amplified by graph propagation.
Questions for authors
-
How did you choose the feature readout for each baseline? Would the strongest comparisons improve if they could use both readouts?
-
How does BIDETA behave with uneven class counts or a smaller query batch? Does matching the query class proportions to the support set still help in those cases?
-
Since the simpler gates are already very close to the full model, would a fixed recurrence weight work just as well?
-
For the small drops on TacTip and ViTac with 1% support, can you tell whether the graph step helps or hurts the memory predictions?
3 comments
3. Can a fixed recurrence weight achieve the same performance?
As discussed in our response to Weakness 2, a fixed weight does not fully retain the gains from query-specific gating at the same mean propagation strength. On SITR with Sparsh and 10% labeled data, keeping all other configurations identical and assigning every query the original gate’s mean weight yields 86.12 ± 0.69% accuracy, compared with 87.09 ± 0.61% for full BIDETA. Query-specific gating improves accuracy by 0.97 percentage points, supporting the benefit of adjusting propagation strength for individual queries.
4. Does graph propagation improve memory predictions?
Graph propagation remains beneficial on average on both sensors. As discussed in our response to Weakness 3, on TacVerse Shape with Sparsh and 1% labeled data, keeping the support sets, query sets, and class-prior normalization identical, adding graph propagation increases mean accuracy from 38.44% to 39.00% on TacTip and from 58.52% to 58.74% on ViTac, yielding gains of 0.56 and 0.22 percentage points, respectively. These results are averaged over three random seeds and support the benefit of graph propagation for memory predictions.
Post Rebuttal Comment
Thanks for the extra experiments and work. The comparisons using the same feature readouts addres my concerns about the baselines. While the gain over the fixed mean weight is still pretty modest, it provides some evidnce for query-specific gating. Please include these results in the final version and make clear how the baselines were adapted and the fixed weight was chosen. I still think this approach can be of interest for the community, so I’ll keep my rating unchanged.
We thank the reviewer for recognizing the practical value of our framework and the breadth of our experimental evaluation. We address the comments and questions in order below.
Weakness
1. The baselines use different feature readouts.
Please see our response to Question 1.
2. The full reliability gate improves prediction.
We believe that the similar results in Table 4 do not negate the contribution of the full gate. Single-signal variants still adjust propagation weights for individual queries. These ablations therefore primarily compare reliability signals and do not establish that a fixed weight can replace query-specific gating.
To test this directly, we fixed the LDA predictions, spectrally corrected graph, and class-prior normalization on SITR with Sparsh and 10% labeled data. Assigning all queries the original gate’s mean weight yields 86.12 ± 0.69% accuracy. Shuffling only the correspondence between weights and queries yields 86.02 ± 0.67%. Full BIDETA achieves 87.09 ± 0.61%, improving accuracy by 0.97 and 1.07 percentage points, respectively. These results show that assigning weights to individual queries improves prediction at the same mean propagation strength, supporting the contribution of the full gate.
3. Causes of difficulty with very few support examples
On these two sensors, the difficulty is already present at the support memory stage, while graph propagation remains beneficial on average across three random seeds.
To distinguish the effects of support memory and graph propagation, we additionally analyzed predictions on TacVerse Shape with Sparsh and 1% labeled data. Using identical support sets, query sets, and class-prior normalization, the mean accuracies without graph propagation are 38.44% and 58.52% on TacTip and ViTac, respectively. Adding graph propagation increases them to 39.00% and 58.74%, yielding gains of 0.56 and 0.22 percentage points. These results show that graph propagation still improves memory predictions on average with only three support examples per class.
Question
1. Does the comparison hold when baselines use the same dual readouts?
Some baselines improve with dual readouts, but BIDETA retains its advantage. The original experiments used predefined readout configurations in our baseline implementations. Methods relying on the source classifier used the final readout to match its input, while SimpleShot and LaplacianShot used the earlier readout to construct class prototypes and query neighborhoods.
To directly examine the effect of readout selection, we conducted additional comparisons using identical frozen dual readouts, support samples, and query splits. We provided the concatenated readouts to the baselines and refitted the corresponding source classifiers for the three adapters while retaining their architectures and training objectives. LaplacianShot was retuned using internal validation on the Mini_3 training set and achieved 39.73% on SITR with Sparsh and 10% labeled data, close to its original 39.87%.
The dual-readout adapter results below report accuracy in percentages as mean ± standard deviation over three random seeds with 10% labeled data.
SITR and TacVerse results average six and five target sensors, respectively. These results show that differences in readout count are insufficient to explain BIDETA’s performance advantage. We will supplement Table 6 and Appendix A.4 with the matched-readout results and implementation details.
2. Does prior matching still help with class imbalance and smaller query batches?
In our additional experiments with imbalanced support class counts and smaller query batches, prior matching still improves mean accuracy. We evaluated SITR with Sparsh and 10% labeled data. The following accuracies are averaged over six target sensors and three random seeds.
The imbalanced-support experiment keeps the total labeling budget unchanged, and full BIDETA outperforms dual-readout LDA by 4.52 percentage points. With query batch sizes of 64 and 256, prior matching improves accuracy by 2.18 and 2.66 percentage points, respectively, showing that its benefits persist with smaller query batches.