acceptodds
Under review as a conference paper at ICLR 2027

Radar as Tokens: Learning Unified Radar-Native Representations

Abstract

Millimeter-wave radar is an increasingly important sensing modality for robust perception under adverse weather (outdoor) and lighting conditions (indoor). However, learning transferable radar representations for diverse indoor/outdoor downstream tasks remains challenging because radar measurements are both noisy and geometrically non-uniform: multipath, sidelobes, and low angular resolution introduce ambiguous returns, while the polar-coordinate representation format induces range-dependent spatial resolution; that is, the further from the antenna, the larger the cross-range resolution cell. Existing radar perception pipelines often reuse LiDAR- or Cartesian-grid representations, leaving downstream models to resolve radar-specific noise and geometry implicitly through task supervision. We introduce (Radar as Tokens), referred to as RadTok, a radar-native, task-agnostic representation. RadTok is built on two complementary principles. First, it adopts task-agnostic discrete spatial tokenization, inspired by VQ-style representation learning, to compress radar tensors into compact tokens that suppress ubiquitous radar clutter while preserving consistent target structure. Second, it introduces radar-native operations through resolution-aware polar-grid resampling and a learned de-sharpener that models radar’s spatially varying degradation. The resulting representation is denoised, resolution-aware, and directly consumed by existing spatial-feature-based perception pipelines without any modification to their architectures. With a single tokenizer design, RadTok consistently improves radar perception across multiple downstream tasks, reducing indoor absolute pose error by 17.8% on RT-Pose and, on K-Radar, outperforming baseline methods by a margin of 10+ AP in outdoor 3D detection and 2.2 mIoU in occupancy grid prediction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Radar, Tokenization, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_W8gKOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RadTok, a radar representation that combines discrete tokenization, range-aware polar resampling, and an inverse-degradation objective. The goal is to suppress radar clutter while reducing the range-dependent resolution variation of polar radar tensors. The method is evaluated on indoor pose estimation and outdoor detection/occupancy prediction using RT-Pose and K-Radar. The reported results show improvements over task-specific baselines and several adapted representation-learning methods.

Critical strengths
  • The paper explicitly targets non-uniform polar resolution and radar clutter/multipath instead of treating radar as a generic image-like input.

  • RadTok can be inserted before existing downstream backbones without changing the task-specific architecture, which makes the design relatively practical.

  • The paper reports results on indoor pose estimation, outdoor 3D detection, and occupancy prediction, with ablations and multiple random seeds, which gives a reasonably broad empirical picture.

Critical weaknesses
  • Cross-sensor generalization is not convincingly addressed. Different FMCW waveforms, antenna layouts, virtual array sizes, and angular resolutions can produce very different radar statistics and artifacts. RadTok does not explicitly model these sensor-dependent factors, and the paper provides no strong zero-shot cross-sensor evaluation. This makes it unclear whether the learned tokenizer captures radar-generic structure or simply adapts to the statistics of the training sensors. The paper itself leaves substantially different sensor configurations for future work.

  • The tokenizer is difficult to assess as a reusable representation without released pretrained weights. Since the main claim is a task-agnostic radar tokenizer, reproducibility should ideally include the pretrained tokenizer/checkpoints, not only the architecture and training recipe. I could not find a clear commitment in the manuscript to release pretrained RadTok weights, which makes the claimed reusable representation hard to independently verify or transfer to other radar datasets.

  • The claimed task-agnostic and transferable representation is not really demonstrated. RadTok is pretrained and then jointly fine-tuned with the task loss separately for each downstream task. There is no convincing cross-task or cross-sensor transfer experiment using one frozen tokenizer. Therefore, the experiments mainly show that the same architecture can be retrained for different tasks, which is much weaker than learning a unified transferable radar representation.

  • The inverse-degradation formulation is severely under-constrained. Both the high-resolution representation and the degradation blur are learned from the same noisy input, while supervision only requires their composition to reconstruct that input. Many different high-resolution representations and blur kernels can satisfy this objective, so there is no guarantee that the recovered feature map corresponds to a physically meaningful clean radar signal.

  • The claim of suppressing sidelobes and multipath is not sufficiently supported. LFQ discretization does not inherently distinguish true targets from sidelobes or multipath, and these artifacts can be stable and highly structured rather than “inconsistent noise.” The paper provides no explicit ghost annotations, PSF measurements, or quantitative sidelobe/multipath suppression evaluation to validate this central claim.

  • The gain from resolution-aware spatialization is confounded with interpolation and increased computation. RadTok upsamples the azimuth dimension by as much as 4×4\times–6×6\times, causing the downstream network to process a substantially larger feature map. The paper does not provide a clean matched-resolution control using simple interpolation or continuous features, making it difficult to tell whether the gain comes from the proposed radar-native representation or simply denser input and more computation. At fmax⁡=4f_{\max}=4, latency increases from 5.8 ms to 36.6 ms and memory from 217 MiB to 980 MiB.

  • Resampling cannot recover missing physical angular information by itself. The method interpolates measurements from a fixed-resolution radar grid and then claims to recover fine-grained structure that the raw input lacks. Without higher-resolution radar ground truth or another physical constraint, this is interpolation rather than demonstrated resolution recovery, and the generated high-resolution structures may be task-driven hallucinations.

  • The main pipeline is not actually token-native. Although the paper is framed as “Radar as Tokens,” the main experiments decode the discrete tokens back into a dense high-resolution feature map before feeding them to existing backbones. Direct use of the tokens is only tested in an appendix experiment on one K-Radar detection setting.

  • The experimental breadth is weaker than the “unified” claim suggests. There are only two radar datasets; two of the three downstream tasks come from the same K-Radar dataset. The paper itself leaves substantially different sensor configurations for future work, so cross-sensor generalization is largely untested.

Questions for authors
  1. Is the same pretrained RadTok tokenizer transferred across tasks or sensors, or is it retrained/fine-tuned separately for each setting? If the latter, what evidence supports the “task-agnostic” and “transferable” claims?
  2. How does RadTok handle changes in FMCW waveform, antenna layout, virtual aperture, and angular resolution across radar platforms? Do you have any zero-shot cross-sensor results with a frozen tokenizer?
  3. Since both the high-resolution representation and the degradation blur are learned from the same noisy input, what prevents non-unique or non-physical solutions that still reconstruct the observation well?
  4. Can you provide quantitative evidence that the tokenizer actually suppresses sidelobes and multipath, rather than simply improving downstream features?
  5. Have you compared against a matched-resolution baseline using the same range-dependent upsampling but without discrete tokenization or inverse degradation? This seems necessary to isolate where the gain comes from.
  6. The main experiments decode tokens back into dense features before the downstream backbone. What advantage does the discrete tokenizer provide over a continuous bottleneck under matched capacity and resolution?
Reviewer_UrawOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces RadTok, a task-agnostic method for learning denoised, resolution-aware token representations from 4D millimeter-wave radar tensors. It combines a lookup-free quantization (LFQ) bottleneck, intended to suppress inconsistent noise such as ghost echoes, with range-dependent azimuth resampling that compensates for the non-uniform cell size of the polar grid. The tokenizer is pre-trained with a self-supervised inverse-degradation objective that requires no paired clean data, then fine-tuned jointly with the downstream task. The resulting representation can be fed to existing perception backbones without architectural changes. It improves performance on indoor human pose estimation (RT-Pose) and on outdoor 3D object detection and occupancy prediction (K-Radar).

Critical strengths
  • The method clearly improves over the task-specific baselines on all three benchmarks. These span indoor and outdoor settings, different radar hardware, and different downstream architectures.

  • The learned representation is a drop-in replacement for the input of existing spatial-feature-based radar perception pipelines. It requires no changes to the backbone or task head, which makes it practical to adopt.

  • The ablation studies (Tables 2–4) isolate the contribution of each component, the resampling factor, and the quantization scheme, which supports the paper’s design choices.

Critical weaknesses
  • Section 3.2 makes two central claims about the discrete bottleneck. The first is that “signals with consistent spatial structure are preserved by the discrete bottleneck, while transient noise and multipath artifacts are suppressed.” The second is that “the bottleneck stabilizes the tokens against ghost echoes, sidelobes, and clutter while preserving recurring, physically meaningful structure.” Neither claim is supported by theoretical analysis. The paper also does not clearly explain why the described method (a convolutional encoder followed by sign quantization) would have these properties. The ablations show that LFQ improves downstream performance, but they do not establish that the improvement comes from this noise-suppression mechanism.

  • The transferability claims aren’t tested. The abstract and related work claim transferable representations that generalize across platforms and tasks. However, the tokenizer appears to be pre-trained separately per dataset and fine-tuned per task, with no cross-dataset transfer experiment.

Questions for authors
  • Following up on the weakness above: is there a theoretical basis for these claims, or are they supported only by empirical results? If the latter, could the authors provide more direct evidence that the bottleneck specifically suppresses ghost echoes and multipath artifacts?
Reviewer_Lu93OpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RadTok is a task-agnostic radar representation learning framework designed to address radar-specific noise, ghost echoes, and range-dependent spatial resolution. Its main contributions are (1) discrete radar tokenization to suppress inconsistent noise and ghost echoes, (2) resolution-aware polar-grid resampling to better account for nonuniform spatial resolution, and (3) a self-supervised inverse-degradation objective that learns to recover cleaner, higher-resolution representations without paired clean radar data. The framework is evaluated on three downstream tasks—indoor human pose estimation, outdoor 3D object detection, and occupancy prediction—using RT-Pose and K-Radar. Results show improvements over baseline methods, including a 17.8% reduction in pose absolute error, gains of over 10 AP in 3D detection, and a 2.2-point increase in occupancy mIoU. Ablation studies examine the contributions of the proposed components, while runtime and memory measurements assess computational overhead.

Critical strengths
  1. Addresses two important, radar-specific challenges explicitly at the representation level rather than leaving them entirely to downstream task supervision.

  2. Demonstrates consistent quantitative improvements across three downstream tasks, with substantial gains in K-Radar 3D detection (+10.2–10.7 AP) and RT-Pose absolute pose error (17.8% reduction).

  3. Provides component ablations and computational-cost analysis, supporting the contributions of tokenization and resolution-aware resampling.

Critical weaknesses
  1. Limited conceptual novelty: The approach largely combines established techniques—discrete quantization, spatial resampling, learned deblurring, and self-supervised reconstruction—with relatively straightforward radar-specific adaptations.

  2. Limited evaluation breadth: Experiments cover only two datasets and three tasks, with outdoor detection and occupancy both evaluated on K-Radar. Generalization across radar hardware, sensor configurations, and unseen environments remains unverified.

  3. Unclear representation-learning benefits: The method is pretrained and fine-tuned on each downstream task, making it difficult to establish the strength of its task-agnostic transferability. The inverse-degradation objective also lacks direct validation that the reconstructed high-resolution features correspond to physically accurate radar structure rather than task-useful artifacts.

Questions for authors

no

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.