acceptodds
Under review as a conference paper at ICLR 2027

LEMMO: A Large Electromagnetic Model Leveraging Pretrained LLMs for Million-Sample Signal Understanding

Abstract

Large language models have expanded beyond text to vision, audio, and other modalities, but understanding raw electromagnetic (EM) signals remains challenging. Wideband recordings can contain millions of samples, whereas existing waveform-based EM models are primarily evaluated on short in-phase and quadrature (IQ) segments, leaving long-recording understanding and detailed description insufficiently explored. We introduce LEMMO, a Large EM MOdel for instruction-driven recognition and description of long raw IQ recordings. LEMMO combines a linear-attention EM encoder with anti-aliased temporal resampling to process up to eight million complex samples. Its EM-text interaction module conditions learnable queries on instructions before retrieving signal evidence. Compact queries support recognition, while structured queries with auxiliary factual supervision organize scene-level properties, signal attributes, and cross-signal relationships for detailed description. We also construct and release a unified 1.38-TB corpus of real-world and synthetic EM signals, including LEMMO-ShortQA and LEMMO-LongDesc. Compared with MERLIN-LoRA, LEMMO improves mean EM-Bench perception accuracy by 15.77 %, mean strategy-generation ROUGE-L F1 by 4.38 points, and mean LEMMO-ShortQA accuracy by 14.31 %. It also improves categorical and numerical fact F1 on both LEMMO-LongDesc tasks while preserving its language backbone's MMLU performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Electromagnetic Model, MLLM, Electromagnetic Signal Processing and Understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_y5Z3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LEMMO, an electromagnetic signal-language model designed to process raw IQ recordings containing up to eight million samples. It combines a linear-attention encoder, temporal resampling, and instruction-conditioned queries to support signal recognition and detailed descriptions. The authors also introduce two datasets, LEMMO-ShortQA and LEMMO-LongDesc. Experiments show improvements over MERLIN-LoRA on recognition and description tasks, while preserving the language model's general knowledge performance.

Critical strengths

Extending electromagnetic signal understanding to long recordings. Most existing waveform-based EM-language models focus on relatively short signals. LEMMO addresses a meaningful limitation by supporting recordings with millions of IQ samples and introducing tasks that require understanding multiple signal components and their relationships. The new datasets also provide a useful starting point for studying more complex EM-language understanding.

Critical weaknesses

1. The main methodological contributions are not properly validated.

The paper introduces instruction-conditioned queries, structured query groups, and auxiliary factual supervision as key contributions. However, there are no controlled ablations showing whether these components actually improve performance. For example, it is unclear whether Q128 performs better than ordinary learnable queries, or whether the auxiliary fact losses provide any benefit. More importantly, the EM-Bench experiments do not even use the proposed query modules, making those results irrelevant to validating the central methodological claims. Without these comparisons, it is difficult to determine what the proposed method contributes beyond existing architectures.

2. The main long-recording comparison is fundamentally confounded by unequal input access.

On LEMMO-LongDesc, LEMMO processes the complete recording, whereas MERLIN-LoRA only receives the first 4,096 IQ samples. Since the tasks require identifying signal components and relationships across long recordings, the baseline may simply lack the information needed to answer correctly. The reported improvements therefore do not establish that LEMMO's proposed architecture is better. A comparison with baselines given access to the same recording is necessary to support the central claim.

Questions for authors
  1. How much do instruction-conditioned queries and structured factual supervision contribute individually? Could the authors compare Q128 against ordinary learnable queries with the same number of tokens, both with and without auxiliary fact losses?

  2. Can the authors evaluate a baseline that has access to the complete IQ recording, rather than only the first 4,096 samples? This would help determine whether the improvements come from the proposed architecture or simply from having access to more signal information.

4 comments
AuthorsOpenReview ↗

Author Response (1/2)

Thank you for recognizing the value of extending electromagnetic signal understanding to long recordings. We agree that the original evaluation lacked sufficient component ablations and that unequal recording access limited the conclusions supported by the MERLIN-LoRA comparison. We address these concerns with component ablations and a full-recording baseline using the same language backbone, evaluated on the Multi-signal Scene Overview task of LEMMO-LongDesc. This task requires identifying multiple signal components and describing their attributes and relationships. The additions are included in the revised manuscript and marked in blue.

Response to Weakness 1 and Question 1: Contributions of the proposed components

(1) Component ablations of LEMMO.

We evaluate four variants of LEMMO:

  • w/o Aux. removes auxiliary factual supervision.
  • w/o Struct. replaces Q128 structured queries with 128 ordinary learnable queries, preserving the query-token budget.
  • w/o Instr. removes instruction-conditioned retrieval.
  • w/o Temp. replaces the Temporal Resampler with adaptive average pooling.

Cat. F1 and Val. F1 denote Category F1 and Value F1, respectively; Avg. is their arithmetic mean.

Table 1. Component ablations on Multi-signal Scene Overview.

\text{Configuration} & \text{R-1} & \text{R-2} & \text{R-L} & \text{BLEU-4} & \text{Token F1} & \text{Cat. F1} & \text{Val. F1} & \text{Avg.} \\\\ \hline \text{w/o Aux.} & 74.66 & 64.48 & 69.80 & 44.57 & 72.31 & 42.54 & \mathbf{44.93} & 43.74 \\\\ \text{w/o Struct.} & 74.54 & 64.34 & 69.63 & 44.91 & 72.55 & \mathbf{44.90} & 44.42 & 44.66 \\\\ \text{w/o Instr.} & 73.38 & 62.68 & 68.14 & 50.57 & 71.41 & 39.85 & 41.63 & 40.74 \\\\ \text{w/o Temp.} & 74.73 & 64.41 & 69.59 & 44.44 & 72.40 & 42.96 & 44.02 & 43.49 \\\\ \text{LEMMO} & \mathbf{76.45} & \mathbf{65.43} & \mathbf{70.83} & \mathbf{50.94} & \mathbf{74.80} & 44.83 & 44.83 & \mathbf{44.83} \end{array}$$ The complete LEMMO configuration achieves the highest score on every text metric. It also ranks second individually in Cat. F1 and Val. F1, while achieving the highest average fact score. The largest reduction (9.1%) occurs when instruction-conditioned retrieval is removed, consistent with its role in selecting question-relevant evidence from long recordings. Regarding the question about Q128 and ordinary learnable queries, compared with w/o Struct, complete LEMMO improves BLEU-4 from 44.91 to 50.94 and Token F1 from 72.55 to 74.80, and achieves a higher average fact score. As for the auxiliary fact losses, the result of the w/o Aux. configuration shows that removing these losses reduces BLEU-4 from 50.94 to 44.57 and the average fact score from 44.83 to 43.74. ***(2) Why EM-Bench uses a direct projection interface rather than Q16/Q128.*** Q16 Recognition and Q128 Description provide compact signal interfaces for the long recordings in LEMMO-ShortQA and LEMMO-LongDesc. Their fixed query budgets prevent the language-model input from being dominated by a large number of EM tokens. Note that EM-Bench recordings contain only 512 or 1,024 IQ samples, with a maximum length of 1,024. This issue therefore does not arise on EM-Bench, so we retain all features and use a direct projection interface as described in Appendix B.4.
AuthorsOpenReview ↗

Author Response (2/2)

Response to Weakness 2 and Question 2: Full-recording comparison with the same language backbone

We agree that the original comparison cannot, by itself, establish an architectural advantage: a baseline restricted to the first 4,096 samples may lack evidence needed for scene-wide descriptions. Although this cropping follows the official MERLIN implementation and is documented in Appendix C.2, it remains a limitation of the comparison.

To address this concern, we construct M-full, a MERLIN-LoRA adaptation that receives the complete IQ recording through a learnable compression module. For recordings longer than 4,096 samples, this module maps the full input to 4,096 samples before processing by MERLIN-LoRA. We additionally replace its language backbone with Qwen3.5-4B, matching LEMMO, and jointly train the compression module, projector, DSM, and Qwen3.5 LoRA parameters.

Table 2 compares this adaptation with LEMMO and M-original, the original MERLIN-LoRA configuration using the first 4,096 samples and Qwen3-4B-Instruct-2507.

Table 2. Full-recording baseline comparison on Multi-signal Scene Overview.

\text{Model} & \text{R-1} & \text{R-2} & \text{R-L} & \text{BLEU-4} & \text{Token F1} & \text{Cat. F1} & \text{Val. F1} & \text{Avg.}\\\\ \hline \text{M-original} & 72.76 & 61.37 & 66.17 & 49.43 & 70.31 & 39.76 & 36.10 & 37.93 \\\\ \text{M-full} & 73.59 & 60.56 & 68.10 & 44.14 & 71.83 & 43.71 & \mathbf{45.34} & 44.53 \\\\ \text{LEMMO} & \mathbf{76.45} & \mathbf{65.43} & \mathbf{70.83} & \mathbf{50.94} & \mathbf{74.80} & \mathbf{44.83} & 44.83 & \mathbf{44.83} \end{array}$$ LEMMO outperforms M-full on all five text metrics, including 6.80 points in BLEU-4 and 2.97 points in Token F1. It also achieves higher Category F1 by 1.12 points, while average factual F1 is 0.30 points higher. These results establish LEMMO’s advantage over this full-recording baseline when both use Qwen3.5-4B.
AuthorsOpenReview ↗

Gentle Reminder

Dear reviewer,

Thank you for your valuable comments and constructive suggestions.

As the discussion deadline is approaching, we would appreciate your participation in the discussion so that we have sufficient time to address any further questions or concerns you may have.

If you feel that our response has addressed some of your concerns, would you kindly consider raising your rating score?

Thank you for your time and consideration, and we look forward to hearing from you.

Best Regards,

Authors of LEMMO

AuthorsOpenReview ↗

Gentle Follow-up

Dear Reviewer,

As the discussion period will end in a few hours, we would like to kindly follow up on our previous response and welcome any remaining questions or concerns you may have.

If our response has addressed some of your concerns, we would sincerely appreciate it if you could consider updating your rating score.

Thank you again for your valuable time and consideration. We look forward to hearing from you.

Best regards,

Authors of LEMMO

Reviewer_ukdLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LEMMO, a framework for instruction-driven electromagnetic signal recognition and natural-language description from long raw IQ recordings. The model combines a linear-attention encoder with a temporal compression module, processes up to eight million complex samples, and feeds the resulting signal representations into Qwen3.5-4B. For recognition tasks, the authors use 16 instruction-conditioned queries with auxiliary classification supervision. For detailed description, they use 128 queries organized into global, object, and relation groups, together with auxiliary factual supervision. The model is adapted through staged training and joint fine-tuning. The authors also construct LEMMO-ShortQA and LEMMO-LongDesc to evaluate multi-class signal recognition and fine-grained description capabilities. Under the reported experimental settings, LEMMO achieves notable improvements over MERLIN-LoRA and other methods across multiple tasks.

Critical strengths
  1. A clearly motivated problem and novel system integration

In electromagnetic signal recognition and natural-language description, high sampling rates produce large inputs even over short observation periods, while short-window analysis may miss burst activity, recurring events, and relationships across time. Beyond identifying signal categories, this paper attempts to describe the attributes and relationships of multiple signals in long recordings. This poses meaningful challenges for signal encoding, long-sequence compression, and instruction-relevant information extraction. I believe that combining long raw IQ processing with natural-language analysis is a problem worth studying.

I believe that the novelty of this paper lies mainly in its problem formulation for long raw IQ recordings and the corresponding system integration. To address long-recording processing and multi-signal description, the authors combine long-sequence encoding and compression with instruction-conditioned retrieval, and design different query organizations and supervision mechanisms for recognition and description. These designs correspond clearly to the task requirements and form a relatively complete research framework, offering a valuable exploration of combining LLMs with complex electromagnetic signal analysis.

  1. Useful data resources and broad initial evaluation

The authors compile and construct a corpus of approximately 1.38 TB of real-world and synthetic electromagnetic signals, with corresponding datasets for recognition and long-recording description. The paper provides data sources, sizes, splits, and task definitions, along with anonymous links to the code and data. These resources and task settings provide a useful reference for future research combining electromagnetic signals with language models.

The experiments cover signal perception, strategy generation, and long-recording description, rather than focusing on a single classification problem. Under the comparison settings used, LEMMO improves upon MERLIN-LoRA on multiple metrics. The description tasks report both text-similarity metrics and metrics for categorical and numerical facts, allowing the generated responses to be evaluated from different perspectives. These results provide preliminary evidence for the framework's overall effectiveness and a basis for further analysis of its signal-understanding capabilities.

Critical weaknesses
  1. Insufficient evidence for the contributions and relative advantages of the proposed design

Core-module ablations

The paper proposes instruction-conditioned queries, structured query grouping, auxiliary factual supervision, and temporal compression, but the existing experiments do not sufficiently isolate their contributions. Table 8 shows how results change with input length, but it cannot replace architectural ablations with the input held fixed or separate the benefits of expanded observation coverage from those of the specific architectural design. In addition, Appendix B.4 states that EM-Bench uses a direct encoder-to-LLM projection rather than Q16/Q128. Improvements on this benchmark therefore cannot directly validate the contributions of instruction-conditioned or structured queries.

I would like to see a small number of controlled experiments addressing the core questions. For example, with the input, language backbone, and number of queries held constant, the authors could compare variants with and without instruction conditioning, structured query grouping, and auxiliary factual supervision. They could also compare the proposed temporal compression scheme with simple compression or aggregation alternatives. These comparisons would help clarify what benefits the specific designs provide over simpler approaches.

Language-model baselines and comparison conditions

Some general-purpose LLMs obtain zero or near-zero scores on recognition and fact-level description metrics. However, the paper does not sufficiently explain how these models receive and process IQ, or whether the low scores are affected by input representation, prompt settings, or answer parsing. These results therefore do not adequately represent what general-purpose LLMs could achieve with appropriate signal-processing support.

I believe that tool-augmented LLM or agent baselines should also be considered alongside the current general-purpose LLM comparisons. For example, models could call signal-processing tools or generate answers from structured evidence extracted by tools from the same recording. Such comparisons could help determine what additional benefits the proposed jointly trained architecture provides over a "signal analysis tools + LLM" pipeline. These baselines could initially be limited to one or two representative tasks with established signal-analysis procedures; a general-purpose agent covering all tasks is not required.

In addition, LEMMO and MERLIN-LoRA use different language backbones. On LongDesc, LEMMO processes complete recordings, whereas MERLIN-LoRA processes only the first 4,096 samples. The ability to use complete recordings is itself a strength of this paper, but the current comparison changes both the model architecture and the amount of observable information, making their respective contributions difficult to distinguish. I suggest adding at least one baseline that covers the complete recording and reporting its computational cost. Representative experiments with the same language backbone would also help assess the effect of backbone differences.

Comparisons with specialized methods and performance-cost trade-offs

The paper studies natural-language description while also evaluating problems with established task definitions, such as modulation recognition and wireless technology classification. However, the main comparisons focus on LLM-related methods and lack representative non-LLM specialized methods evaluated under the same data splits and input conditions. Without such comparisons, it is not possible to determine how introducing a language model changes performance and computational costs on these basic tasks.

I recognize the value of natural-language interaction and multi-task coverage, and I do not require the framework to outperform specialized models on every task. Accepting some performance loss in certain settings to obtain broader capabilities can be reasonable, but this trade-off needs to be quantified and discussed.

The authors should select representative recognition or parameter-estimation tasks and compare their approach with traditional signal-processing methods or non-LLM task-specific models under the same conditions. The comparison should clarify which aspects of performance are retained, where gaps remain, and how computational costs change. Even if the results show lower performance on some tasks, such an analysis would help clarify the method's applicability and would not automatically negate its research value. The current lack of evidence makes it difficult for readers to determine when LEMMO should be adopted and when specialized methods are sufficient.

  1. Insufficient validation of training text and specification of factual evaluation

The authors state in the AI Use Statement that generative AI assisted in constructing the QA text for ShortQA and LongDesc, and that the generated content was manually reviewed. The appendix specifies the fixed question phrasings and target labels for ShortQA, and describes the factual supervision and some structured reference answers for LongDesc. However, for the complex descriptions and numerical facts in LongDesc, the paper does not sufficiently explain how factual labels are obtained, how text generation is constrained, or how human review checks consistency between the text and the waveforms.

Since these texts are used for training, I am concerned about the extent to which the model learns to recover electromagnetic facts from IQ, as opposed to fitting the generative model's linguistic style, answer templates, and content distribution. If the training and test reference texts use similar generation procedures and templates, text-similarity metrics may not adequately distinguish between these possibilities. Although the paper introduces fact-level metrics, the correspondence between ground-truth facts and generated text also remains insufficiently explained.

In particular, does the LLM rewrite text based on existing structured ground truth, or does it also infer factual content? What criteria guide human review, and how does it check whether numerical values, event counts, temporal boundaries, and object relationships are consistent with the actual signals? How are descriptions unsupported by annotations or waveform evidence identified and handled? At present, a statement of human review alone is insufficient to assess the reliability of this part of the training supervision.

Category F1 and Value F1 in the main results are currently explained primarily at a conceptual level. The paper does not sufficiently explain how facts are extracted from free-form text, how objects are matched in multi-signal cases, how units are normalized, or what matching tolerances are used for different numerical attributes. These rules directly affect the interpretation and reproducibility of the core results.

Questions for authors
  1. Regarding the LLM-generated training text

Please provide further details on the QA text generation process, particularly how complex factual descriptions in LongDesc are generated, which models and prompt settings are used, and where each type of fact comes from. Is the LLM only responsible for converting structured ground truth into natural language? What is the coverage of human review, what criteria are used, and how are errors corrected? Are there review statistics or examples demonstrating consistency between the text and the signals?

  1. Regarding the contributions of the core modules

With the input, language backbone, and query budget held fixed, how much does each of instruction conditioning, structured query grouping, and auxiliary factual supervision contribute? What advantages does the temporal compression module offer over simple compression or multi-window aggregation? Could you provide a small number of ablation results that directly validate the main designs?

  1. Regarding the fact-level metrics

How are Category F1 and Value F1 computed from free-form text? Please specify the rules for fact extraction, object matching, and handling omissions and additional generated content, as well as unit normalization and numerical tolerances. Please also provide a complete scoring example.

  1. Regarding the language-model baselines

How exactly do general-purpose LLMs receive the signals, and are their zero or near-zero scores related to limitations in input representation, input length, or output parsing? Have tool-supported LLMs/agents or "signal analysis tools + LLM" pipelines been evaluated on representative tasks? Do LEMMO's advantages still hold after controlling for language-backbone differences and allowing the baselines to observe complete recordings?

  1. Regarding the trade-offs with traditional methods

Under the same data splits and observation conditions, how does LEMMO compare with non-LLM specialized methods in accuracy, estimation error, and computational cost on representative recognition or parameter-estimation tasks? If performance losses occur on certain tasks, in which settings do the authors consider them acceptable, and what additional value do natural-language interaction and multi-task capabilities provide?

7 comments
AuthorsOpenReview ↗

Author Response (1/5)

Thank you for your time and constructive feedback.

For clarity, we summarize your concerns and their corresponding questions as follows:

Weakness 1: Insufficient evidence for the contributions and relative advantages of the proposed design

  • 1.1 Core-module ablations — Question 2.
  • 1.2 Language-model baselines and comparison conditions — Question 4.
  • 1.3 Comparisons with specialized methods and performance-cost trade-offs — Question 5.

Weakness 2: Insufficient validation of training text and specification of factual evaluation

  • 2.1 Training-text validation — Question 1.
  • 2.2 Factual-evaluation specification — Question 3.

We provide detailed responses to your concerns and questions below. A revised manuscript has been uploaded, with added content marked in blue.

1. Response to Question 1 and Weakness 2.1: Training-text construction and factual reliability

Thank you for asking us to clarify the construction and verification of the QA text for LEMMO-ShortQA and LEMMO-LongDesc. The electromagnetic facts originate from waveform-generation metadata or annotations of recorded signals. The text-generation model expresses these facts in natural language and does not estimate additional facts from I/Q signals. We explain the sources, generation settings, review criteria, and correction procedures below.

(1) Where the reference facts come from.

We distinguish the four-task LEMMO-ShortQA benchmark from the targeted QA within LEMMO-LongDesc. In LEMMO-ShortQA, each recording is paired with four fixed question phrasings and its recognition label. LEMMO-LongDesc includes targeted questions about individual facts and long-form scene descriptions. Its facts have the following sources.

Table 1. Sources and derivation of reference facts in LEMMO-LongDesc.

CategoryLabel sources and derivation
Basic modulationSimulation records provide modulation, timing, frequency, and bandwidth; component power and in-band noise determine SNR.
4G LTESimulation labels and carrier layouts provide scene, carrier, spectrum, duplex, band, and SNR attributes.
5G NRConfigurations and schedules provide carrier and resource labels; receiver overlap determines visible bandwidth, and aligned 10-ms boundaries determine frame counts.
ISMWi-Fi/Bluetooth generator records provide frame, channel, protocol, user, SNR, and overlap labels.
UAVAcquisition settings and verified annotations provide statistics of FCS and VTS, device, protocol, and modulation labels.

For all scenes above, components are ordered by start time, center frequency, and end time and assigned identifiers S1S_1, …, SnS_n. The program compares their temporal and spectral intervals to distinguish time-only overlap, frequency-only overlap, and overlap on both axes. Signal identities and relationships are therefore established before text generation.

(2) How the reference text is generated.

LongDesc QA and descriptions share structured facts. Targeted QA applies question-specific fields and premise checks, with templates and rules for LTE, NR, and ISM. For all five signal families, the teacher model, DeepSeek V4 Flash 0731, generates long-form descriptions and is instructed to follow the supplied facts without adding unsupported information.

The teacher receives the question, acquisition settings, and selected structured facts, without the IQ waveform. Prompts specify required attributes, description order, units, and numerical precision, while allowing variation in wording. The table below provides a representative prompt.

Table 2. A representative reference-generation prompt.

Signal familyExample generation prompt
UAVDescribe UAV-related RF presence and FCS/VTS link status, followed by the identified protocol and possible UAV model. Summarize FCS segment counts, time–frequency ranges, durations, spectrum-access method, and modulation, then describe available VTS attributes. Preserve annotated values and certainty while varying the wording.
AuthorsOpenReview ↗

Author Response (2/5)

(3) Reference text checks.

All QA texts underwent automated checks against their source labels or annotations, with manual checks of uncertain cases. Checks covered the fields requested by each question or required by each description prompt:

  • Counts and categories: consistency with source records.
  • Numerical consistency: correct values, unit conversions, rounding, and event boundaries.
  • Attribute binding and relationships: correct assignment to signals or links and correct relation pairs, with FCS and VTS checked separately.
  • Supported content: adherence to annotated facts and correct handling of question premises, including noise-only recordings and absent objects.

Task-specific checks also covered identifiers, formatting, description order, and NR reference-signal consistency with link direction. This process ties each reference answer to its source facts and checks its consistency within the scope of the question or description prompt.

2. Response to Question 2 and Weakness 1.1 : Core-module ablations

(1) Core-module ablations of LEMMO.

Thank you for suggesting these comparisons. We add four component ablations on the Multi-signal Scene Overview task from LEMMO-LongDesc. All configurations receive the same complete IQ recordings, acquisition settings, and questions, use Qwen3.5-4B as the language backbone, and retain a query budget of 128.

The four configurations are defined as follows:

  • w/o Aux. removes auxiliary factual supervision.
  • w/o Struct. replaces Q128 structured queries with 128 ordinary learnable queries.
  • w/o Instr. removes instruction-conditioned retrieval.
  • w/o Temp. replaces the Temporal Resampler with average pooling.

In particular, w/o Temp. retains the same compressed feature length as the Temporal Resampler.

Table 3. Core-module ablations of LEMMO. Cat. F1 denotes Category F1; Val. F1 denotes Value F1; Avg. denotes their arithmetic mean.

\text{Configuration} & \text{R-1} & \text{R-2} & \text{R-L} & \text{BLEU-4} & \text{Token F1} & \text{Cat. F1} & \text{Val. F1} & \text{Avg.} \\\\ \hline \text{w/o Aux.} & 74.66 & 64.48 & 69.80 & 44.57 & 72.31 & 42.54 & \mathbf{44.93} & 43.74 \\\\ \text{w/o Struct.} & 74.54 & 64.34 & 69.63 & 44.91 & 72.55 & \mathbf{44.90} & 44.42 & 44.66 \\\\ \text{w/o Instr.} & 73.38 & 62.68 & 68.14 & 50.57 & 71.41 & 39.85 & 41.63 & 40.74 \\\\ \text{w/o Temp.} & 74.73 & 64.41 & 69.59 & 44.44 & 72.40 & 42.96 & 44.02 & 43.49 \\\\ \text{LEMMO} & \mathbf{76.45} & \mathbf{65.43} & \mathbf{70.83} & \mathbf{50.94} & \mathbf{74.80} & 44.83 & 44.83 & \mathbf{44.83} \end{array}$$ The complete model achieves the highest scores on all five text metrics and the highest average factual F1. Removing instruction-conditioned retrieval produces the largest decrease in average factual F1: 4.09 points, or 9.1% relative, with both Category F1 and Value F1 declining. Removing auxiliary factual supervision reduces average factual F1 by 1.09 points and BLEU-4 by 6.37 points. Replacing Q128 structured queries with 128 ordinary learnable queries reduces BLEU-4 by 6.03 points and Token F1 by 2.25 points, while average factual F1 decreases by 0.17 points. Thus, the aggregate evidence for structured query grouping is stronger for description quality than for factual accuracy. Replacing the Temporal Resampler with simple average pooling reduces all five text metrics, including BLEU-4 by 6.50 points and Token F1 by 2.40 points. Category F1 and Value F1 decrease by 1.87 and 0.81 points, respectively, reducing average factual F1 by 1.34 points. This comparison supports the Temporal Resampler over simple compression at the same compressed feature length and query budget. ***(2) Why EM-Bench uses a direct projection interface rather than Q16/Q128*** Q16 Recognition and Q128 Description provide compact signal interfaces for the long recordings in LEMMO-ShortQA and LEMMO-LongDesc. Their fixed query budgets prevent the language-model input from being dominated by a large number of EM tokens. In contrast, EM-Bench recordings contain only 512 or 1,024 IQ samples, with a maximum length of 1,024. This issue therefore does not arise on EM-Bench, so we retain all features and use a direct projection interface as described in Appendix B.4.
AuthorsOpenReview ↗

Author Response (3/5)

3. Response to Question 3 and Weakness 2.2 : Fact-level metrics and scoring rules

(1) Category F1

The Category F1 implementation normalizes Unicode and letter case, then extracts categorical labels using a predefined set of patterns. These include technology and modulation labels, frequency-band identifiers, duplex modes, link directions, SNR levels and so on. Extracted facts are represented as a multiset. Precision is the number of matched category occurrences divided by the number of generated occurrences; recall uses the number of reference occurrences as its denominator. Category F1 is their harmonic mean.

(2) Value F1

The Value F1 scorer first extracts the predicted value from the answer based on a predefined set of patterns, including the number of signals, start/end times, center frequencies, bandwidths, overlap relationships and so on. The scoring rules are:

  • A matching value contributes one true positive.
  • A missing value contributes a false negative.
  • An additional value contributes a false positive.

For each answer, Value F1 is computed as follows:

F1=2TP2TP+FP+FN.F_1=\frac{2TP}{2TP+FP+FN}.

(3) Unit normalization and numerical tolerances

The value names specify the units, such as milliseconds for time and megahertz for frequency. For integer values, the predicted value must exactly match the reference value to be considered correct. For floating-point values, the error must not exceed one percent. For strings, the answer is considered correct if it matches after text normalization and tokenization.

(4) Complete scoring example

The sample 78 of basic_modulation/iq_parquet/part-00013.parquet contains 1,600,000 IQ samples over 20 ms. The reference specifies three signals from S1S_1 to S3S_3 with modulation labels '16APSK', '16PAM', and 'AM-SSB-SC', and SNR levels 'High', 'Low', and 'Medium', respectively. LEMMO correctly predicts the signal count, all three SNR assignments, and the modulation of S3S_3, while predicting incorrect modulation labels for S1S_1 and S2S_2. The reference and prediction each contain seven scored facts: one signal count, three modulation labels, and three object-specific SNR assignments. Category F1 is computed over the three modulation labels and three object-specific SNR assignments. Four of these six facts are correctly matched, giving precision and recall of 4/64/6 and a Category F1 of 66.67%. Value F1 is computed over the signal count, which is correctly predicted as three, yielding 100%.

4. Response to Question 4 and Weakness 1.2 : Language-model baselines and full-recording comparison

(1) How general-purpose LLMs receive the signals

On EM-Bench, the LLMs receive every IQ sample in temporal order as text, together with the original question and the sampling rate where available. The recordings in EM-Bench contain 512 or 1,024 IQ samples. On the LEMMO-ShortQA and LEMMO-LongDesc datasets, recordings can be up to several million samples long. Therefore, feeding the full-length recordings to LLMs in text form is no longer feasible. Consequently, the tests employ a “signal analysis tool + LLM” pipeline. A NumPy tool reads the IQ file and supplies a fixed textual representation consisting of a time-domain overview and time–frequency features, together with the acquisition settings and question. The temporal overview partitions the complete record into 64 bins for signals of at most 4,096 samples and 96 bins for longer signals, reporting RMS amplitude, peak amplitude, and mean I/Q values. The time–frequency representation contains up to 32 temporal bins and 64 frequency bins, computed from up to four evenly spaced Hann-windowed FFTs per temporal bin, with 3-dB power quantization. Thus, it spans the recording rather than using only its first 4,096 samples. But at the same time, it should be noted that these inputs are limited summaries, not lossless representations of the full recordings. Their time-domain/frequency-domain resolution and spectral sampling windows may affect the information accessible to LLMs.

The output of LEMMO, Merlin-LoRA and four general-purpose LLMs used for comparison is limited to 2,048 tokens. During testing, LEMMO, Merlin-LoRA, GPT-6 and Gemini had no incomplete responses, while DeepSeek had 5, and Qwen had 60 out of 802 cases. All of these responses were included in the scoring.

Reviewer_HDsbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LEMMO, a raw-IQ electromagnetic signal-to-language framework intended to process recordings up to eight million complex samples. It claims three contributions:

  1. Long-signal modeling supporting up to eight million complex samples;
  2. Instruction-conditioned queries with task-specific auxiliary supervision for recognition and description;
  3. Signal–text resources including LEMMO-ShortQA and LEMMO-LongDesc. The core approach combines linear-attention signal encoding with temporal compression and query-based aggregation to convert long IQ sequences into a compact representation for language generation.Evaluation compares LEMMO against MERLIN-LoRA and general-purpose LLMs on EM-Bench, LEMMO-ShortQA, and LEMMO-LongDesc.
Critical strengths

The paper identifies a meaningful gap between conventional EM recognition benchmarks and the demands of understanding complex wireless environments. The problem formulation in §1 moves beyond classifying isolated signal snippets to interpreting continuous recordings where signal locations and frequency bands are unknown in advance. LEMMO-LongDesc operationalizes this setting through questions about signal components, physical attributes, and temporal–spectral relationships (§4.1).

Critical weaknesses
  1. The evaluation can not isolate the contribution of the proposed architecture from the benefits of accessing substantially more signal information and using a different language backbone. Appendix C.2 states that LEMMO processes complete recordings, whereas MERLIN-LoRA receives only the first 4,096 IQ samples while being evaluated on descriptions requiring scene-wide information. The models also use different language backbones—Qwen3.5-4B versus Qwen3-4B-Instruct-2507 (Table 4). No full-recording chunk-and-aggregate baseline or component ablations. Table 8 only varies LEMMO’s input length, while the EM-Bench configuration explicitly bypasses Q16/Q128 and uses a direct projection interface (Appendix B.4).
  2. I am not fully convinced by the method design itself.The overall architecture combines long-sequence encoding, temporal compression, instruction-conditioned queries, and auxiliary factual supervision into a reasonable engineering pipeline. However, it lacks sufficiently targeted mechanisms for the central challenges of fine-grained understanding of long recordings: preserving sparse events, correctly associating attributes with individual signals, and maintaining consistency across signal relationships.
Questions for authors

Beyond enabling long IQ sequences to enter an LLM, what evidence demonstrates that the proposed designs address challenges specific to understanding long recordings? In particular, do structured queries, instruction-conditioned retrieval, and auxiliary factual supervision improve attribute binding, brief-event recognition, or cross-segment reasoning? — Controlled evidence linking these designs to meaningful improvements on these challenges would strengthen my assessment of the methodological contribution and overall score.

5 comments
AuthorsOpenReview ↗

Author Response (1/3)

Thank you for your time and constructive feedback. We address your concerns below with a full-recording baseline using the same language backbone, component comparisons, and concrete examples of attribute binding, brief-event recognition, and cross-segment reasoning. A revised manuscript has been uploaded, with added content marked in blue.

1. Response to Weakness 1: Recording coverage, backbone choice, and component contributions

(1) Full-recording comparison with the same language backbone.

The original MERLIN-LoRA comparison follows its official implementation, which crops inputs longer than 4,096 IQ samples to their first 4,096 samples, as described in Appendix C.2 of the original submission. We acknowledge that this limits access to evidence required for scene-wide descriptions, while the different language backbones introduce an additional confound.

We therefore construct M-full, a MERLIN-LoRA adaptation that receives the complete recording through a learnable compression module mapping it to 4,096 samples. We also replace its language backbone with Qwen3.5-4B, matching LEMMO, and jointly train the compression module, projector, DSM, and Qwen3.5 LoRA parameters. We evaluate this adaptation on the Multi-signal Scene Overview task of LEMMO-LongDesc, which requires descriptions of multiple signals across a recording.

In Table 1, M-original denotes the original 4,096-sample configuration with Qwen3-4B-Instruct-2507. Cat. F1 and Val. F1 denote Category F1 and Value F1; Avg. is their arithmetic mean.

Table 1. Full-recording baseline comparison.

\text{Model} & \text{R-1} & \text{R-2} & \text{R-L} & \text{BLEU-4} & \text{Token F1} & \text{Cat. F1} & \text{Val. F1} & \text{Avg.}\\\\ \hline \text{M-original} & 72.76 & 61.37 & 66.17 & 49.43 & 70.31 & 39.76 & 36.10 & 37.93 \\\\ \text{M-full} & 73.59 & 60.56 & 68.10 & 44.14 & 71.83 & 43.71 & \mathbf{45.34} & 44.53 \\\\ \text{LEMMO} & \mathbf{76.45} & \mathbf{65.43} & \mathbf{70.83} & \mathbf{50.94} & \mathbf{74.80} & \mathbf{44.83} & 44.83 & \mathbf{44.83} \end{array}$$ M-full increases average factual F1 from 37.93 to 44.53, substantially narrowing the original gap. This underscores the importance of controlling recording coverage and backbone choice. Compared with M-full, LEMMO achieves higher scores on all five text metrics, including gains of 6.80 points in BLEU-4 and 2.97 points in Token F1. It also achieves the highest Cat. F1 and the highest average fact score. These results establish LEMMO’s advantage over this full-recording baseline when both use the same language backbone, Qwen3.5-4B. ***(2) Component ablations of LEMMO.*** We agree that Table 8 examines input length rather than individual design choices. We add four component ablations on the same Multi-signal Scene Overview task: w/o Aux. removes auxiliary factual supervision; w/o Struct. replaces Q128 structured queries with 128 ordinary learnable queries; w/o Instr. removes instruction-conditioned retrieval; and w/o Temp. replaces the Temporal Resampler with average pooling. Table 2. Component ablations of LEMMO. $$\begin{array}{l|rrrrr|rrr} \text{Configuration} & \text{R-1} & \text{R-2} & \text{R-L} & \text{BLEU-4} & \text{Token F1} & \text{Cat. F1} & \text{Val. F1} & \text{Avg.} \\\\ \hline \text{w/o Aux.} & 74.66 & 64.48 & 69.80 & 44.57 & 72.31 & 42.54 & \mathbf{44.93} & 43.74 \\\\ \text{w/o Struct.} & 74.54 & 64.34 & 69.63 & 44.91 & 72.55 & \mathbf{44.90} & 44.42 & 44.66 \\\\ \text{w/o Instr.} & 73.38 & 62.68 & 68.14 & 50.57 & 71.41 & 39.85 & 41.63 & 40.74 \\\\ \text{w/o Temp.} & 74.73 & 64.41 & 69.59 & 44.44 & 72.40 & 42.96 & 44.02 & 43.49 \\\\ \text{LEMMO} & \mathbf{76.45} & \mathbf{65.43} & \mathbf{70.83} & \mathbf{50.94} & \mathbf{74.80} & 44.83 & 44.83 & \mathbf{44.83} \end{array}$$ The complete model achieves the highest scores on all five text metrics and the highest average factual F1. Removing instruction-conditioned retrieval produces the largest decrease in average factual F1: 4.09 points, or 9.1% relative, with both Category and Value F1 declining. Removing auxiliary factual supervision or replacing the Temporal Resampler reduces average factual F1 by 1.09 and 1.34 points, respectively. Replacing structured queries with ordinary learnable queries reduces BLEU-4 by 6.03 points and Token F1 by 2.25 points, but average factual F1 decreases by only 0.17 points. Thus, the aggregate evidence for structured queries is stronger for description quality than for factual accuracy. These results support the contributions of each component.
AuthorsOpenReview ↗

Author Response (2/3)

(3) Why EM-Bench uses a direct projection interface rather than Q16/Q128

Q16 Recognition and Q128 Description provide compact signal interfaces for the long recordings in LEMMO-ShortQA and LEMMO-LongDesc. Their fixed query budgets prevent the language-model input from being dominated by a large number of EM tokens. In contrast, EM-Bench recordings contain only 512 or 1,024 IQ samples, with a maximum length of 1,024. This issue therefore does not arise on EM-Bench, so we retain all features and use a direct projection interface as described in Appendix B.4.

2. Response to Weakness 2 and Questions: Design mechanisms for fine-grained understanding of long recordings

Thank you for highlighting the need to connect our design choices to the specific challenges of long-recording understanding. LEMMO’s components are designed to support three complementary functions: retaining brief-event evidence during compression, organizing attributes around individual signals, and representing relationships between signals.

Q128 comprises 128 learnable queries: 16 global queries, 96 object queries organized into 16 groups of six, and 16 relation queries. These queries are intended to represent scene-level properties and signal counts, signal-specific attributes, and cross-object relationships, respectively. They operate jointly with the Electromagnetic Encoder, Temporal Resampler, instruction-conditioned retrieval, and auxiliary factual supervision:

  • Preserving sparse events. The Electromagnetic Encoder extracts local waveform features before temporal compression. The Temporal Resampler combines a low-pass downsampling path with a gated learnable convolutional path, allowing short-duration feature variations to contribute to the compressed representation. Object-query groups can subsequently attend to localized signal activity. The attention visualization in Appendix D.1 illustrates this behavior for a brief RF burst.

  • Binding attributes to individual signals. Grouping object queries provides an organizational structure for retrieving signal-specific features, while auxiliary factual supervision guides attribute predictions and their associations with individual signals. Instruction-conditioned retrieval further guides evidence selection according to the requested information.

  • Representing temporal–spectral relationships. Relation queries provide capacity for cross-object context, complemented by auxiliary supervision of temporal and spectral relationships. This design encourages predictions that account for relationships among signals, although it does not explicitly guarantee relational consistency.

The component ablations reported above provide quantitative evidence of contributions to scene-description performance. We next examine their connection to attribute binding, brief-event attention, and cross-segment relationship recovery through concrete examples.

3. Response to the Questions: Evidence specific to long-recording understanding

(1) Attribute binding: associating SNR levels with the correct signals

The test recording data/basic_modulation/iq_parquet/part-00012.parquet, sample 23, contains 600,000 IQ samples spanning 30 ms, with three signals labeled S1S_1–S3S_3 in order of occurrence. The following table compares the predicted SNR assignments.

\text{Attribute} & \text{Ground Truth} & \text{LEMMO} & \text{MERLIN-LoRA} & \text{w/o Aux.} & \text{w/o Struct.} & \text{w/o Instr.} & \text{w/o Temp.} \\\\ \hline \text{Number of signals} & 3 & 3 & 3 & 3 & 3 & 3 & 3 \\\\ \text{SNR of $S_1$} & \text{High} & \mathbf{High} & \text{Low} & \text{Low} & \text{Low} & \text{Low} & \text{Low} \\\\ \text{SNR of $S_2$} & \text{Low} & \mathbf{Low} & \text{Medium} & \text{Medium} & \text{Medium} & \text{Medium} & \text{Medium} \\\\ \text{SNR of $S_3$} & \text{Medium} & \mathbf{Medium} & \text{High} & \text{High} & \text{High} & \text{High} & \text{High} \end{array}$$ All configurations recover the correct signal count and the same set of SNR levels. The distinction is ***which SNR level is assigned to which signal***: LEMMO correctly associates all three levels with their corresponding signals, whereas MERLIN-LoRA and all four ablated configurations misassign all three. This example illustrates an attribute-binding error that would be missed by evaluating signal counts or attribute values without their signal associations. The comparison is consistent with the intended contribution of the complete design to signal-specific attribute organization, while remaining a qualitative case study.
AuthorsOpenReview ↗

Author Response (3/3)

(2) Brief-event evidence: localized Q128 attention to a short burst

Sample T0000_D00_S0111/RF0/17 from uav/DroneRFa_original_decoupled/test-00000.parquet contains a brief RF burst near 16.7 ms amid low-energy background. Within the 13.6–20 ms close-up, an object query in the third object group exhibits a localized increase in temporal attention near the burst, while much of the surrounding background receives lower attention. The full-recording spectrogram, burst close-up, and corresponding attention heatmap are provided in Appendix D.1 of the revised manuscript. This visualization illustrates that Q128 can attend to brief activity within a longer recording, consistent with the intended role of object queries in retrieving localized evidence.

(3) Cross-segment relationships: relating signals separated in time

The recording data/basic_modulation/iq_parquet/part-00032.parquet, sample 159, contains 600,000 IQ samples spanning 30 ms and five signals, S1S_1–S5S_5. The reference intervals place S1S_1 at 1.18–4.17 ms and S2S_2 at 11.09–14.18 ms. These signals are separated by a 6.92-ms gap, but their occupied frequency bands overlap. Identifying this relationship requires associating spectral information from distinct temporal segments.

\text{Scene fact} & \text{Ground Truth} & \text{LEMMO} & \text{MERLIN-LoRA} & \text{w/o Aux.} & \text{w/o Struct.} & \text{w/o Instr.} & \text{w/o Temp.} \\\\ \hline \text{Number of signals} & 5 & \mathbf{5} & 3 & 3 & 3 & 3 & 3 \\\\ \text{Frequency-only overlap} & S_1\text{--}S_2 & \mathbf{S_1\text{--}S_2} & \text{None} & \text{None} & \text{None} & \text{None} & \mathbf{S_1\text{--}S_2} \\\\ \text{Time-only overlap} & \text{None} & \text{None} & \text{None} & \text{None} & \text{None} & \text{None} & \text{None} \end{array}$$ LEMMO correctly recovers both the five-signal count and the frequency-only overlap between $S_1$ and $S_2$. MERLIN-LoRA and the w/o Aux., w/o Struct., and w/o Instr. variants report only three signals and miss the overlap. The w/o Temp. variant identifies the overlapping pair but still undercounts the signals. Thus, only the complete configuration correctly recovers both facts in this example.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.