acceptodds
Withdrawn from ICLR 2027

Bio-Inspired Events and WiFi for Privacy Preserving HAR: A Large-scale Dataset and Novel Approach

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Human Activity Recognition (HAR) supports applications ranging from healthcare monitoring to human-robot interaction. Existing HAR methods predominantly fuse RGB cameras and WiFi signals. However, RGB cameras raise privacy concerns, introduce redundancy, and degrade in low-light or occluded scenes, while WiFi lacks the spatial resolution to resolve fine-grained motion. Bio-inspired event cameras capture sparse brightness changes with high temporal resolution and reduced appearance detail, motivating a central research question: How can asynchronous event streams be effectively integrated with heterogeneous wireless signals to achieve robust and privacy-preserving HAR under real-world conditions? To investigate this question, we introduce ExWACT, the first large-scale event-WiFi HAR dataset, comprising 22.5K paired samples across 6 scenes in challenging simulated and real-world scenarios. By pairing complementary event streams and WiFi CSI, ExWACT enables systematic evaluation of multimodal fusion under diverse sensing conditions and investigation of domain shifts between simulated and real event data. Based on the dataset, we propose a novel temporally consistent fusion framework. Our key idea is to bridge the modality gap between dense wireless signals and sparse, asynchronous event streams via event-WiFi interaction and temporal alignment. The framework comprises two components. A Sparse Dual-branch Modality Encoder extracts motion-sensitive representations tailored to WiFi CSI and event streams. A Temporally Consistent Bi-directional Fusion (TCBF) module enables dynamic interaction between WiFi and events to capture latent spatial-temporal cues while enforcing temporal alignment. This design maintains high HAR accuracy and robustness under challenging visual conditions where traditional RGB-WiFi systems fail. Extensive experiments demonstrate that our method significantly outperforms pure WiFi baselines by +45.48% and existing fusion strategies by +13.65%. ExWACT establishes a new direction for privacy-preserving HAR, bridging wireless WiFi and bio-inspired sensing and laying a solid foundation for further research on real-world HAR.

Primary area:
datasets and benchmarks
Keywords:
Multimodal learning, Human activity recognition, Event-based vision, WiFi sensing, Cross-modal fusion, Temporal alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BzPHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies human activity recognition through the fusion of event cameras and WiFi CSI. The authors construct the ExWACT dataset, which includes simulated event data converted from WiMANS videos and paired real-world event and WiFi data collected with an event camera. The method uses modality-specific dual-branch encoders, bidirectional cross-attention, and a temporal consistency loss to fuse features from the two modalities.

Critical strengths

(1) The paper explores joint sensing with event data and WiFi and provides both simulated and real paired data, which supports the study of the complementary properties of these two sensing modalities. (2) The real-world data include different lighting conditions, background human motion, and partial occlusion, which gives the dataset some practical relevance, as described in Section 3.1 and Appendix A.

Critical weaknesses

(1) Although event data reduce dense appearance information, they can still preserve recoverable identity-related features, and some studies discuss the reconstruction of recognizable faces from event streams. Therefore, the event representation itself does not provide a sufficient guarantee of anonymization. The paper does not directly evaluate identity recognizability, so the statement in Table 1 that event data require no post-processing for anonymization lacks direct support. At the same time, the paper does not report a complete quantitative comparison among RGB, RGB-WiFi fusion, and event-WiFi fusion under the same conditions. It therefore remains unclear what benefits and costs result from replacing RGB in terms of recognition accuracy and robustness to low light and occlusion. This makes the motivation for choosing the event modality insufficiently supported. (2) The main operations in Section 3.2 include temporal differencing, convolutional encoding, event voxelization, polarity processing, bidirectional cross-attention, and representation-distance constraints. Much of the novelty appears to lie in the combination of event data and WiFi and the integration of these operations for this task, while the independent algorithmic contribution that specifically addresses the heterogeneous characteristics of the two modalities is not sufficiently clear. (3) Figure 4 does not specify the RGB model, training data, or testing protocol. The RGB predictions also include a rotation class that does not appear in the class list of the real-data branch. It is therefore necessary to confirm whether the two systems use the same task definition. (4) In Figure 14, the diagonal entries of the confusion matrix for THAT on the real-data branch sum to 225 out of 315 samples, which corresponds to an accuracy of approximately 71.43%, while Table 3 reports 78.73%. In the same figure, DECOLLE gives 307/315, or 97.46%, which is consistent with Table 3. If Figure 14 uses a different test set or model version, this difference should be clearly stated. (5) The two projections in Eq. 6 and Eq. 13 in the appendix are both learnable functions, and both can output zero to make the consistency loss equal to zero. Under the current definition, the classification objective does not exclude this degenerate solution. The decrease in accuracy after removing this loss supports its empirical usefulness, but it does not directly demonstrate effective temporal alignment.

Questions for authors

(1) What is the basis for choosing the event modality over RGB? Can the authors provide comparisons of RGB-WiFi fusion and event-WiFi fusion under the same data conditions and with matched models, including accuracy and performance under low-light and occlusion conditions? What direct evidence supports the claim that event data require no anonymization? (2) Compared with standard modality encoding, bidirectional cross-attention, and representation alignment, what new design specifically addresses the heterogeneity between event data and WiFi? Does a standard fusion method with the same encoders and training data pool still perform clearly worse? (3) Do the temporal consistency projections share parameters with the classification path, or are there additional constraints that are not described in the paper? How does the actual implementation prevent both projections from outputting zero and losing their alignment effect?

Reviewer_oW4sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ExWACT, a multimodal human activity recognition (HAR) dataset combining WiFi Channel State Information (CSI) and event-camera measurements. The dataset contains approximately 22.5K paired samples across six scenes, combining simulated event streams derived from existing RGB-WiFi recordings with newly collected real-world event-camera data. The authors also propose a multimodal recognition framework featuring modality-specific encoders, bidirectional cross-attention, and a temporal consistency loss. Experiments report 93.74% overall classification accuracy, outperforming the evaluated unimodal and multimodal baselines. Additional experiments investigate component contributions, challenging sensing conditions, and cross-scene and cross-domain generalization.

Critical strengths
  1. Potentially valuable multimodal dataset. The combination of event-camera measurements and WiFi CSI addresses an interesting sensing problem. Including both simulated and real-world recordings, together with challenging illumination and occlusion conditions, could make ExWACT a useful benchmark for future research. The newly collected synchronized real-world measurements are particularly valuable.

  2. Strong reported performance with component-level validation. The proposed framework achieves higher accuracy than the evaluated unimodal and multimodal baselines. The ablation studies indicate that the modality-specific encoders, bidirectional attention, and temporal consistency objective each contribute to performance. The additional cross-scene and cross-domain experiments strengthen the empirical evaluation.

Critical weaknesses
  1. Limited methodological novelty and insufficiently competitive multimodal baselines. The proposed framework primarily combines established techniques, including modality-specific encoding, cross-attention, and representation alignment. Although their adaptation to event-WiFi sensing is reasonable, the methodological advance is relatively limited. More importantly, all three evaluated multimodal competitors are author-implemented combinations of existing components, and their overall accuracies (77.57–80.09%) are substantially below that of the strongest event-only baseline (88.78%). This raises concerns about whether the claimed improvement over multimodal fusion methods reflects a genuinely superior fusion mechanism or insufficiently competitive baselines.

  2. Dataset validation and privacy claims require stronger evidence. Although the paper advertises 22.5K paired samples, a substantial portion consists of simulated event streams converted from an existing RGB-WiFi dataset. The newly collected real-world subset involves only 10 participants, and the paper does not sufficiently clarify whether the evaluation splits prevent subject- or recording-level overlap. Such details are particularly important given the near-saturated real-world accuracy of 99.18%. Additionally, the claimed privacy-preserving properties are not quantitatively evaluated. Event cameras reduce the amount of appearance information captured, but this does not by itself establish privacy preservation.

Questions for authors
  1. Please clarify the exact dataset composition and the subject-level separation between the training and test sets.

  2. Multimodal baseline competitiveness: Why do all three multimodal fusion baselines perform substantially worse than the strongest event-only model? Were their encoder backbones, optimization procedures, and training budgets matched appropriately? Could the authors provide a stronger fusion baseline that incorporates the EvT event encoder and demonstrates that adding WiFi improves performance beyond event-only recognition?

  3. Contribution of WiFi: Given the strong event-only performance, can the authors quantify the incremental benefit of WiFi under matched architectures and training conditions, particularly in low-light and occluded scenarios? This would help establish whether WiFi provides meaningful complementary information rather than merely increasing model capacity.

  4. Privacy preservation: What evidence supports the privacy-preserving claim beyond the reduced appearance information associated with event cameras? For example, can the authors evaluate identity leakage or the recoverability of identifying information from the event representations? If such evaluation is outside the paper's scope, the privacy claims should be qualified accordingly.

Reviewer_WjHMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new research topic by combining the event camera and wifi signals for privacy-preserving HAR. A benchmark dataset and a new baseline model is proposed for the task. The baseline seems a simple alignment between the two modalities. The organization of this paper is good and the paper is well-written.

Critical strengths
  1. the task is new and the idea is interesting
  2. the paper is well written and well organized
  3. the experimental results seem validated
Critical weaknesses

1.The checkmark symbol should be set to green in the Table 1; 2.The following event-based HAR benchmark datasets should be introduced and discussed in Table 1 and corresponding paragraphs, e.g., HARDVS, CeleX-HAR. 3.The latest research articles published in year 2025 and 2026 should be also introduced in the section 2. 4.The dataset features in challenging scenarios like low-light and dynamic occlusion conditions, however, the motion related factors are seldom to consider. For example, the motion speed of different human activities can also be treated as a major aspect. 5.The compared method in Table 2 is not comprehensive and the latest algorithms published in year 2025 and 2026 are none.

Questions for authors

1.The checkmark symbol should be set to green in the Table 1; 2.The following event-based HAR benchmark datasets should be introduced and discussed in Table 1 and corresponding paragraphs, e.g., HARDVS, CeleX-HAR. 3.The latest research articles published in year 2025 and 2026 should be also introduced in the section 2. 4.The dataset features in challenging scenarios like low-light and dynamic occlusion conditions, however, the motion related factors are seldom to consider. For example, the motion speed of different human activities can also be treated as a major aspect. 5.The compared method in Table 2 is not comprehensive and the latest algorithms published in year 2025 and 2026 are none.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.