- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LEMMO, an electromagnetic signal-language model designed to process raw IQ recordings containing up to eight million samples. It combines a linear-attention encoder, temporal resampling, and instruction-conditioned queries to support signal recognition and detailed descriptions. The authors also introduce two datasets, LEMMO-ShortQA and LEMMO-LongDesc. Experiments show improvements over MERLIN-LoRA on recognition and description tasks, while preserving the language model's general knowledge performance.
Critical strengths
Extending electromagnetic signal understanding to long recordings. Most existing waveform-based EM-language models focus on relatively short signals. LEMMO addresses a meaningful limitation by supporting recordings with millions of IQ samples and introducing tasks that require understanding multiple signal components and their relationships. The new datasets also provide a useful starting point for studying more complex EM-language understanding.
Critical weaknesses
1. The main methodological contributions are not properly validated.
The paper introduces instruction-conditioned queries, structured query groups, and auxiliary factual supervision as key contributions. However, there are no controlled ablations showing whether these components actually improve performance. For example, it is unclear whether Q128 performs better than ordinary learnable queries, or whether the auxiliary fact losses provide any benefit. More importantly, the EM-Bench experiments do not even use the proposed query modules, making those results irrelevant to validating the central methodological claims. Without these comparisons, it is difficult to determine what the proposed method contributes beyond existing architectures.
2. The main long-recording comparison is fundamentally confounded by unequal input access.
On LEMMO-LongDesc, LEMMO processes the complete recording, whereas MERLIN-LoRA only receives the first 4,096 IQ samples. Since the tasks require identifying signal components and relationships across long recordings, the baseline may simply lack the information needed to answer correctly. The reported improvements therefore do not establish that LEMMO's proposed architecture is better. A comparison with baselines given access to the same recording is necessary to support the central claim.
Questions for authors
-
How much do instruction-conditioned queries and structured factual supervision contribute individually? Could the authors compare Q128 against ordinary learnable queries with the same number of tokens, both with and without auxiliary fact losses?
-
Can the authors evaluate a baseline that has access to the complete IQ recording, rather than only the first 4,096 samples? This would help determine whether the improvements come from the proposed architecture or simply from having access to more signal information.
4 comments
Author Response (2/2)
Response to Weakness 2 and Question 2: Full-recording comparison with the same language backbone
We agree that the original comparison cannot, by itself, establish an architectural advantage: a baseline restricted to the first 4,096 samples may lack evidence needed for scene-wide descriptions. Although this cropping follows the official MERLIN implementation and is documented in Appendix C.2, it remains a limitation of the comparison.
To address this concern, we construct M-full, a MERLIN-LoRA adaptation that receives the complete IQ recording through a learnable compression module. For recordings longer than 4,096 samples, this module maps the full input to 4,096 samples before processing by MERLIN-LoRA. We additionally replace its language backbone with Qwen3.5-4B, matching LEMMO, and jointly train the compression module, projector, DSM, and Qwen3.5 LoRA parameters.
Table 2 compares this adaptation with LEMMO and M-original, the original MERLIN-LoRA configuration using the first 4,096 samples and Qwen3-4B-Instruct-2507.
Table 2. Full-recording baseline comparison on Multi-signal Scene Overview.
\text{Model} & \text{R-1} & \text{R-2} & \text{R-L} & \text{BLEU-4} & \text{Token F1} & \text{Cat. F1} & \text{Val. F1} & \text{Avg.}\\\\ \hline \text{M-original} & 72.76 & 61.37 & 66.17 & 49.43 & 70.31 & 39.76 & 36.10 & 37.93 \\\\ \text{M-full} & 73.59 & 60.56 & 68.10 & 44.14 & 71.83 & 43.71 & \mathbf{45.34} & 44.53 \\\\ \text{LEMMO} & \mathbf{76.45} & \mathbf{65.43} & \mathbf{70.83} & \mathbf{50.94} & \mathbf{74.80} & \mathbf{44.83} & 44.83 & \mathbf{44.83} \end{array}$$ LEMMO outperforms M-full on all five text metrics, including 6.80 points in BLEU-4 and 2.97 points in Token F1. It also achieves higher Category F1 by 1.12 points, while average factual F1 is 0.30 points higher. These results establish LEMMO’s advantage over this full-recording baseline when both use Qwen3.5-4B.Gentle Reminder
Dear reviewer,
Thank you for your valuable comments and constructive suggestions.
As the discussion deadline is approaching, we would appreciate your participation in the discussion so that we have sufficient time to address any further questions or concerns you may have.
If you feel that our response has addressed some of your concerns, would you kindly consider raising your rating score?
Thank you for your time and consideration, and we look forward to hearing from you.
Best Regards,
Authors of LEMMO
Gentle Follow-up
Dear Reviewer,
As the discussion period will end in a few hours, we would like to kindly follow up on our previous response and welcome any remaining questions or concerns you may have.
If our response has addressed some of your concerns, we would sincerely appreciate it if you could consider updating your rating score.
Thank you again for your valuable time and consideration. We look forward to hearing from you.
Best regards,
Authors of LEMMO
Author Response (1/2)
Thank you for recognizing the value of extending electromagnetic signal understanding to long recordings. We agree that the original evaluation lacked sufficient component ablations and that unequal recording access limited the conclusions supported by the MERLIN-LoRA comparison. We address these concerns with component ablations and a full-recording baseline using the same language backbone, evaluated on the Multi-signal Scene Overview task of LEMMO-LongDesc. This task requires identifying multiple signal components and describing their attributes and relationships. The additions are included in the revised manuscript and marked in blue.
Response to Weakness 1 and Question 1: Contributions of the proposed components
(1) Component ablations of LEMMO.
We evaluate four variants of LEMMO:
Cat. F1 and Val. F1 denote Category F1 and Value F1, respectively; Avg. is their arithmetic mean.
Table 1. Component ablations on Multi-signal Scene Overview.
\text{Configuration} & \text{R-1} & \text{R-2} & \text{R-L} & \text{BLEU-4} & \text{Token F1} & \text{Cat. F1} & \text{Val. F1} & \text{Avg.} \\\\ \hline \text{w/o Aux.} & 74.66 & 64.48 & 69.80 & 44.57 & 72.31 & 42.54 & \mathbf{44.93} & 43.74 \\\\ \text{w/o Struct.} & 74.54 & 64.34 & 69.63 & 44.91 & 72.55 & \mathbf{44.90} & 44.42 & 44.66 \\\\ \text{w/o Instr.} & 73.38 & 62.68 & 68.14 & 50.57 & 71.41 & 39.85 & 41.63 & 40.74 \\\\ \text{w/o Temp.} & 74.73 & 64.41 & 69.59 & 44.44 & 72.40 & 42.96 & 44.02 & 43.49 \\\\ \text{LEMMO} & \mathbf{76.45} & \mathbf{65.43} & \mathbf{70.83} & \mathbf{50.94} & \mathbf{74.80} & 44.83 & 44.83 & \mathbf{44.83} \end{array}$$ The complete LEMMO configuration achieves the highest score on every text metric. It also ranks second individually in Cat. F1 and Val. F1, while achieving the highest average fact score. The largest reduction (9.1%) occurs when instruction-conditioned retrieval is removed, consistent with its role in selecting question-relevant evidence from long recordings. Regarding the question about Q128 and ordinary learnable queries, compared with w/o Struct, complete LEMMO improves BLEU-4 from 44.91 to 50.94 and Token F1 from 72.55 to 74.80, and achieves a higher average fact score. As for the auxiliary fact losses, the result of the w/o Aux. configuration shows that removing these losses reduces BLEU-4 from 50.94 to 44.57 and the average fact score from 44.83 to 43.74. ***(2) Why EM-Bench uses a direct projection interface rather than Q16/Q128.*** Q16 Recognition and Q128 Description provide compact signal interfaces for the long recordings in LEMMO-ShortQA and LEMMO-LongDesc. Their fixed query budgets prevent the language-model input from being dominated by a large number of EM tokens. Note that EM-Bench recordings contain only 512 or 1,024 IQ samples, with a maximum length of 1,024. This issue therefore does not arise on EM-Bench, so we retain all features and use a direct projection interface as described in Appendix B.4.