- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes PrivFusionNet to address identity leakage in EEG and eye movement fusion and applies it to emotion recognition. The method combines privacy-aware fusion, task–identity disentanglement, and a discrete codebook to suppress identity information in fused representations. The experimental results show that the method reduces user identification accuracy while maintaining emotion recognition performance, with additional validation through cognitive load recognition experiments.
Critical strengths
- The paper tries to address a practically relevant privacy issue, namely whether privacy protection applied separately to each modality remains effective after fusion.
Critical weaknesses
- The contribution is incremental and limited, as the main proposed modules are based on a combination of existing methods.
- The ablation experiments are unclear and should not be confined to the appendix. In addition to evaluating the effects of removing each module, the effects of using each module alone should also be evaluated. The paper also lacks ablations within each module. The effects of key module parameters, such as the number of rank components R in PA-LRF and the codebook size M in FPCB, should also be evaluated. The paper also does not explain why these parameter values were chosen.
Questions for authors
- The privacy evaluation uses a fixed classifier architecture and training budget, without examining how these choices affect the results. How sensitive are the reported privacy gains to these choices, and what limitations should be considered when interpreting the results?
- On SEED-IV, removing either PA-LRF or FTID produces only a small change in fused UIA. What evidence supports their complementary roles and the explanation that FPCB masks their contributions?
- The results show reduced identity leakage, but provide limited evidence explaining why the proposed modules suppress re-leakage. Can the authors provide experimental evidence to clarify how these modules work and what this tells us about the causes of re-leakage?
- Appendix B.2 states that the “best-performing model” is selected, but does not specify the validation split or selection metric. How is the validation set constructed from the training data, and which metric is used for model selection?
4 comments
Response to Reviewer 2Se9 (Part 2/2: Q2–Q4)
Response to Q2: Small ablation effects and complementarity
The small UIA changes on SEED-IV measure the marginal effects of PA-LRF and FTID when the other modules remain active. In the complete model, FPCB already reduces final fused UIA to near-chance levels, leaving limited room for further measurable reductions. This provides a possible explanation for why removing PA-LRF or FTID produces only a small change in final UIA; the removal results alone do not establish their standalone contributions or demonstrate that FPCB masks their effects.
To examine their contributions more directly, we have added configurations that introduce only PA-LRF, FTID, or FPCB to the same backbone with fusion-level GRL, reporting both BCA and UIA. Together with the existing two-module configurations, these comparisons evaluate the effects of adding PA-LRF or FTID with and without FPCB. They therefore allow us to assess both identity suppression and task-information preservation, rather than inferring complementarity solely from small changes in final UIA. The corresponding results and analysis are provided in Appendix C.
Response to Q3: Evidence for how the modules work
Our working explanation is that complementary residual identity cues can remain in individually protected modalities. Jointly accessing these cues through fusion may make identity information more accessible than it is in either modality alone.
The proposed modules target different stages of this process. PA-LRF downweights rank components with high identity risk during fusion, reducing the contribution of identity-related cues. FTID suppresses identity-related directions after fusion, limiting their retention in the continuous representation. FPCB then constrains fine-grained differences between individuals through subject-shared discrete prototypes. Together, these mechanisms aim to reduce identity leakage while preserving task-relevant information.
The extended module-level ablations in Appendix C enable or disable these mechanisms under matched conditions and report both BCA and UIA. These experiments assess each mechanism's empirical contribution and the effects of their combinations. They provide evidence about how the proposed design affects leakage and utility, while the explanation based on complementary residual identity cues remains a mechanistic interpretation rather than a uniquely established cause.
Response to Q4: Validation split and model selection
The validation set is constructed entirely within the outer training partition. The outer test data are excluded from both model selection and training.
For example, in a LOSO fold on SEED-IV, we first hold out one subject for testing and divide the remaining 14 source subjects into 13 training subjects and one validation subject. During model selection, model weights are updated using only the training subjects' data, and validation performance is used to select the configuration.
After model selection, we merge the validation subject's data into the training partition and complete final training for the predefined 80 epochs using all 14 source subjects under the selected configuration. Validation data therefore participate in final training only after model selection is complete. The outer test subject remains independent and is used solely for final evaluation.
The revised Appendix B.2 describes the validation-set construction and final training procedure, distinguishing validation-based selection from final training on the combined training and validation data.
Thank you for the responses and revisions. My main concern remains that the privacy evaluation relies on a single type of identity classifier. The new Table 7 shows that representations with near-chance UIA can become highly identifiable once simply combined, so near-chance UIA from one per-sample probe is weak evidence of identity removal. Identity may still be revealed by aggregating the patterns of codeword use across multiple samples, which the current evaluation does not test. In addition, the framework largely builds on existing EEG methods (e.g., CMSLNet, ID-RemovalNet, MIND-EEG), so its methodological novelty remains limited. I therefore maintain my rating.
Response to Reviewer 2Se9 (Follow-up: Methodological Contributions and Privacy Evaluation)
Thank you for raising this concern. We would like to clarify that our methodological contribution does not lie in introducing entirely new primitives, but in formulating and addressing a distinct privacy failure mode at the multimodal fusion level. Specifically, PA-LRF introduces identity-risk-aware weighting into low-rank cross-modal interactions through ; FTID explicitly learns an identity-related subspace in the fused representation space and suppresses it through projection, which is fundamentally different from IDS, where identity suppression relies on adversarial training with gradient reversal at the unimodal level; and FPCB combines subject-shared vector quantization with subject-conditional code-usage regularization. These mechanisms differ in their mathematical formulations, optimization objectives, and positions in the processing pipeline, operating at complementary stages of cross-modal interaction, continuous representation, and discrete representation. Their joint design therefore constitutes a fusion-oriented privacy framework rather than a direct reuse of existing EEG privacy methods.
Regarding cross-sample codeword aggregation, we acknowledge that it represents a stronger attack setting than the per-sample identity probe used in our evaluation. However, privacy protection and identity inference inherently involve an evolving interplay between defense and attack capabilities. Our objective is to identify fusion-induced identity re-leakage and establish a systematic defense against identity attacks under a controlled and commonly adopted evaluation protocol, rather than claim immunity to every conceivable attacker. Importantly, FPCB already incorporates subject-conditional code-usage regularization to reduce subject-specific code-selection preferences. More advanced aggregation-based attacks can motivate corresponding improvements to privacy defenses. We view this as an iterative research process: as attack capabilities advance, privacy mechanisms and their evaluations should evolve accordingly, rather than treating the existence of increasingly powerful attacks as invalidating the value of the current defense.
Response to Reviewer 2Se9 (Part 1/2: W1–W2 and Q1)
Thank you for recognizing the practical importance of evaluating whether unimodal privacy protection remains effective after fusion. Below, we clarify our contributions, the extended ablation design, parameter selection, and the scope of the attacker-based evaluation.
Response to W1: Contribution and novelty
Our core contributions lie in identifying fusion-level identity re-leakage and designing a privacy-protection framework tailored to this problem. We empirically investigate how identity information can become accessible after fusing EEG and eye-movement representations, even when identity information has been suppressed within each modality. This finding motivates explicitly evaluating and protecting the fused representation.
Our framework addresses different stages of this process. PA-LRF incorporates identity risk into rank-component weighting, allowing fusion to account for both task information and identity-leakage risk. FTID suppresses identity-related directions in the fusion space while preserving task information. FPCB constrains fused representations through subject-shared discrete prototypes to suppress fine-grained identity cues. These mechanisms are jointly optimized under task and privacy objectives.
To assess the added value of these designs, we have added comparisons with a fusion-level GRL baseline and configurations containing individual modules or module combinations. Reporting both BCA and UIA allows us to evaluate their contributions in terms of identity suppression and task-information preservation.
Response to W2: Ablation design and parameter choices
The original Table 5 evaluates each module's marginal contribution when the other two remain active. We have extended this comparison with configurations that introduce only PA-LRF, FTID, or FPCB to the same backbone with fusion-level GRL, alongside the existing two-module combinations and the complete model. All configurations follow consistent training and evaluation protocols, with both BCA and fused UIA reported. These comparisons distinguish individual module effects from the benefits of combining modules. The accompanying analysis is provided in Appendix C.
Regarding parameter selection, R controls the representational capacity of low-rank fusion in PA-LRF, while M controls the codebook capacity and degree of representation compression in FPCB. Both influence the balance between task-information preservation and identity suppression. Their values are selected using validation performance within the training partition.
For example, in a LOSO fold on SEED-IV, one subject is held out for testing, and the remaining 14 subjects are divided into 13 training subjects and one validation subject. After selecting the parameter configuration, the validation subject's data are merged into the training partition for final training on all 14 source subjects. The outer test subject is excluded from parameter selection and model training.
Response to Q1: Attacker architecture and training budget
Our privacy evaluation follows the identity-classifier-based evaluation paradigm used in studies such as ID-RemovalNet. We independently train a nonlinear MLP identity probe on fixed representations and use the same attacker architecture, training budget, and data splits across all methods in the privacy comparison. This provides a controlled basis for comparing identity leakage across methods.
Attackers with greater capacity, different architectures, or more extensive training may recover additional identity information.
Accordingly, our conclusions concern identity-leakage mitigation in the final fused representation under the evaluated identity attacks. Near-chance UIA under this protocol should not be interpreted as a guarantee that all identity information has been removed.