- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates LLM agents for wireless algorithm discovery under iterative experimentation. The authors find a key gap: existing LLM agents tend to accumulate and tune algorithm components but rarely question whether current accepted components are useful when the surrounding stages change. To address this gap, the authors propose a novel framework, namely BREATHE, which explicitly coordinates parallel exploration with periodic challenges to inherited algorithm components, thereby enabling the agent to develop, reassess, simplify, re-explore, and revise the structure as the algorithm evolves. To evaluate BREATHE’s performance, the authors test four wireless tasks (OFDM synchronization and tracking, channel estimation, channel equalization, MU-MIMO precoding) and three orchestrator models (Qwen3.6-27B, Llama-4-Scout-109B, or GPT-OSS-120B). Experimental results show that BREATHE can achieve better performance than existing agent baselines.
Critical strengths
- The topic of this paper—evaluating components of wireless systems through iterative “parallel exploration” and “component challenging”—is promising for improving communication performance. This topic is timely and of practical value.
- The experiments are comprehensive. The authors consider four wireless tasks (OFDM synchronization and tracking, channel estimation, channel equalization, and MU-MIMO precoding), and employ three orchestrators. Extensive experimental results validate the superiority of the BREATHE framework over existing agent baselines across diverse tasks.
Critical weaknesses
-
The introduction of the BREATHE framework in Section 4 lacks clarity, particularly the implementation details of "component challenge" and “component recheck.” For example, how do we ensure that a candidate edit has removed a component without incurring unintended side effects? The relevant parts are not sufficiently explained.
-
The BREATHE framework relies on LLMs to perform component evaluation and "challenging," rather than explicit mathematical analysis. While LLMs can enhance certain aspects of wireless systems, this framework’s reliability cannot be guaranteed.
Questions for authors
After reading this manuscript, I have four questions as follows:
- Section 3.2 evaluates the performance of four methods—direct LLM, coding agent, AutoResearch, and AI Telco Engineer—on four wireless tasks. How reliable and robust are these experimental results? Given that LLMs can produce different outputs even with the same inputs, have the authors taken this issue into account? Further, how does this issue affect the effectiveness and robustness of the BREATHE framework?
- The introduction of the BREATHE framework in Section 4 is not clear yet. Could the authors clarify the detailed procedures of BREATHE, particularly the “component challenge” and “component recheck” parts? Besides, how does the framework address the limitations of existing AI Telco Engineer agent methods?
- Section 4.2 sets 10-round implementation assignments (3/2/3/2) to successive explore-challenge-reexplore-recheck blocks. Is this setting (3/2/3/2) the optimal one? If not, how do we set the round assignments in practical systems to ensure the effectiveness of BREATHE?
- In Table 2 and Fig. 3, the authors compare the performance of BREATHE and other baseline methods. Although BREATHE achieves the best task performance, the costs (e.g., computational cost and memory usage) are not presented. Did the authors conduct experiments under the same resource constraints?
Additionally, I have some comments on the paper writing and organization:
- The Introduction of the paper is not clear, particularly the third and fourth paragraphs that introduce the paper design and experimental results. Key innovations and results should be highlighted.
- The related work section introduces existing studies of LLM agents for wireless communications, but mainly lists some references without detailed descriptions. Although the authors introduce some works in the Appendix, it is better to clearly introduce important references in the main paper.
- The paper’s organization needs improvement. In Section 3.2, Table 1 summarizes the experimental scaffolding of all evaluated methods, including the proposed BREATHE. However, BREATHE is developed in Section 4. It is unsuitable to present the experimental scaffolding before introducing the framework itself.
- The location of Table 2 is not appropriate. The results of Table 2 are introduced on page 4, but the table is presented on page 7.
3 comments
Response by Authors (Part 2/3)
3. Detailed procedures of BREATHE (Q2 & W1)
We apologize that Section 4 did not describe these procedures clearly. We clarify them below and will revise Section 4 accordingly.
-
Complete BREATHE workflow. BREATHE starts from the basic baseline. In each exploration round, the orchestrator proposes multiple hypotheses and designs experiments to test them, primarily adding components to help the pipeline handle complex signals. The best candidate replaces the incumbent only if it improves performance, and subsequent exploration builds on the incumbent. All other candidates and results are also saved for the orchestrator to review. During challenge, the orchestrator proposes multiple hypotheses about components in the incumbent and designs separate controlled experiments to test their contributions. Each candidate targets one component; the best improving candidate is retained. This block lasts two rounds, with the second starting from the updated incumbent. Re-exploration then develops this incumbent through multiple hypotheses and experiments. Finally, recheck applies the same controlled testing procedure to identify any harmful components remaining in the final solution; no further exploration follows.
-
Avoiding unintended side effects. Before evaluation, the orchestrator reviews each candidate’s code and the worker’s implementation summary against the parent. Removing or replacing a component may change tensor shapes or interfaces at its boundaries, so workers may make the minor adjustments needed to keep the surrounding pipeline executable; these are part of the intervention. Any other change counts as an unintended side effect, and the orchestrator returns the candidate for reimplementation. Each worker has at most two retries, after which its candidate is discarded. Only candidates that pass this check are scored.
Addressing AI Telco Engineer’s limitation. We find that AI Telco Engineer develops algorithms mainly by tuning and recombining components, but does not reassess inherited ones: across 36 trajectories, none of its 1,800 manager proposals suggested removing an inherited component. Component challenge and recheck address this gap by testing inherited components during search and after re-exploration.
4. Optimality of the 3/2/3/2 allocation (Q3)
Under current models, 3/2/3/2 is among the best schedules we tested, and schedules following the same principle perform comparably. What matters is a general principle, which also tells how to set the rounds in practice.
BREATHE alternates exploration and challenge, and each stage needs enough consecutive rounds: (i) enough exploration before challenging, so that there are developed components to reassess; (ii) enough consecutive challenge rounds to reassess inherited components; and (iii) enough exploration after reassessment to develop the revisions. The 3/2/3/2 schedule (EEECCEEECC; E: exploration, C: challenge) is one implementation of this principle for a ten-round budget.
We evaluated alternative ten-round schedules on four tasks with GPT-OSS-120B as the orchestrator. Schedules satisfying (i)–(iii) (EECCEEEECC, EEEECCEECC, and EECCCEECCC) perform comparably to BREATHE, differing by at most 6%, consistent with the Qwen3.6-27B results in Sec. 4.2. Schedules violating them recover only 21–35% of BREATHE's improvement compared with AI Telco Engineer, and trajectory inspection traces each loss to a violated property. The evenly spaced ECECECECEC (21.04%) violates (i) and (ii), and EEEECEEEEC (34.91%) isolates single challenge rounds, violating (ii). Of two random schedules, ECCECCCEEE (29.77%) concentrates challenges early, violating (i), and EEEECECCCC (33.11%) concentrates them late, violating (iii).
In practice, the rounds can be set by following (i)–(iii) instead of searching for an exact optimum. In our experiments, schedules worked well when they kept at least two exploration rounds before the first challenge block, at least two consecutive challenge rounds, and at least two exploration rounds after it. For larger budgets, more explore–challenge cycles can be added following the same principle.
Response by Authors (Part 3/3)
5. Resource constraints and costs (Q4)
All research loops were run under the same resource constraints. BREATHE and the research-loop baselines use matched search budgets of 50 hypotheses and 200 implementation attempts, with the same evaluator and the same per-trial time and memory limits. Since matched implementation budgets do not imply equal token usage or runtime cost, we additionally measured the cost of the discovery process and of the resulting algorithms.
Discovery cost. BREATHE used 15.8% fewer tokens than AI Telco Engineer (106.45M vs. 126.40M) on average for each search.
Cost of the discovered algorithms. We benchmarked the final algorithms on the same RTX A6000 GPU using each task’s original evaluator batch size. The table reports median steady-state latency and incremental peak GPU memory during algorithm execution.
| Task | BREATHE Latency | AI Telco Engineer Latency | BREATHE Memory | AI Telco Engineer Memory |
|---|---|---|---|---|
| Channel Estimation | 50.01 ms | 137.18 ms | 142.24 MiB | 1699.90 MiB |
| Channel Equalization | 3.78 ms | 8.01 ms | 20.51 MiB | 22.84 MiB |
| MU-MIMO Precoding | 0.57 ms | 2.40 ms | 24.63 MiB | 26.85 MiB |
| OFDM Sync./Tracking | 25.42 ms | 203.76 ms | 2.93 MiB | 2.95 MiB |
BREATHE’s final algorithms have lower latency on all four tasks and lower or comparable GPU memory usage, with the largest memory reduction on channel estimation. Together with the token measurements, these results show that BREATHE’s performance gains under matched budgets come with lower discovery and execution costs.
Writing and organization
Thank you for these suggestions. We propose the following revisions.
1. Introduction. We will rewrite paragraphs 3–4 so that the gap, the method, and the results are each stated directly, and highlight the following:
- Gap. Existing agents improve algorithms by tuning and recombining components, but rarely test whether inherited components are still useful: none of 1,800 AI Telco Engineer manager proposals did so.
- Key innovation. BREATHE schedules controlled component challenges between exploration blocks, so that successful revisions seed further exploration.
- Key result. BREATHE achieves the best mean held-out performance in all twelve task–orchestrator comparisons, with a 20.35% macro-average improvement over AI Telco Engineer under the same search budget.
2. Related work. We propose replacing the first paragraph of Related work with this:
LLM agents for wireless communications. TeleQnA evaluates telecommunications knowledge, TelecomGPT develops domain-specific LLMs, and NetLLM adapts LLMs to networking tasks. Other studies address wireless hardware design and network control or replace signal-processing blocks with neural models. For physical-layer algorithm discovery, AI Telco Engineer couples an LLM orchestrator with Sionna to iteratively propose, implement, and evaluate algorithms. Its follow-up extends autonomous design to OTFS and pilot-free OFDM receivers. We analyze whether these searches reassess inherited components as the surrounding pipeline changes. BREATHE introduces controlled component tests and develops successful revisions through further exploration.
Citations will be retained.
3. Table 1 and method ordering. Introducing BREATHE-specific mechanisms before the framework may make the presentation harder to follow. We apologize for this oversight. We will move Table 1 and its definitions of challenge, recheck, and re-exploration to the end of Section 4.2. Section 3.2 will retain only the baseline setup.
4. Table 2 placement. We combined the results to keep comparisons together, as Section 5.1 also uses them for the overall evaluation. However, the reviewer is right that this places the table too far from the earlier discussion. We will split it as shown below: Table A beside Section 3.2 and Table B beside Section 5.1, with a “Best of Table A” row connecting the comparisons.
Table A. Classical methods, LLMs, and coding agents.
| Category | Method | Est. ↓ | Equal. ↑ | Prec. ↑ | OFDM ↑ |
|---|---|---|---|---|---|
| Classical | Basic | ... | ... | ... | ... |
| Strong | ... | ... | ... | ... | |
| LLM | Qwen3.6-27B | ... | ... | ... | ... |
| ... | ... | ... | ... | ... | |
| Coding Agent | Codex (GPT-5.5) | ... | ... | ... | ... |
| ... | ... | ... | ... | ... |
Table B. Research-loop comparison.
| Orchestrator | Method | Est. ↓ | Equal. ↑ | Prec. ↑ | OFDM ↑ |
|---|---|---|---|---|---|
| — | Best of Table A | ... | ... | ... | ... |
| Qwen3.6-27B | AutoResearch | ... | ... | ... | ... |
| AI Telco Engineer | ... | ... | ... | ... | |
| BREATHE | ... | ... | ... | ... | |
| ... | ... | ... | ... | ... | ... |
If you find these revisions suitable, we will incorporate them into the manuscript. We sincerely appreciate the reviewer’s thoughtful comments and constructive guidance, which have helped us refine our work and clarify it.
Response by Authors (Part 1/3)
We sincerely thank the reviewer for their time, careful reading, and constructive comments. We address each concern below.
1. Reliability and robustness of the Section 3.2 results (Q1)
Yes, We took into account that LLMs can produce different outputs even with the same inputs. We handle this at two levels for every method, including BREATHE. Within a search, four workers independently implement each idea (except in the coding-agent baseline), and the best implementation by discovery score represents that idea, reducing reliance on a single implementation. Across searches, we repeat the complete discovery process three times per method, which captures variation in idea generation, implementation, and the orchestrator’s decisions. Each run’s selected algorithm is then frozen and evaluated on 20 unseen seeds.
Under this protocol, BREATHE achieves the best mean among the evaluated research loops in all twelve task–orchestrator comparisons. BREATHE is also more robust: averaged over orchestrators and tasks, its run-to-run standard deviation is 33.9% lower than that of AI Telco Engineer. Furthermore, although individual LLM outputs vary, the agents’ preferences during search remain relatively consistent.
2. LLMs versus mathematical analysis (W2)
BREATHE does not use the LLM as the final judge of whether a wireless algorithm component is useful; it uses the LLM to reason about possible designs, while controlled experiments provide the evidence. Wireless algorithms operate under complex and environment-dependent channel conditions, whose distributions and interactions are often difficult to capture with a tractable analytical model. Consequently, many practical design choices cannot be determined from a closed-form math analysis alone. Instead, human engineers typically combine domain knowledge and approximate reasoning to formulate hypotheses, test them under representative channel conditions, and use experimental feedback to revise the design. BREATHE aims to reproduce this workflow with an agent.
Specifically, the orchestrator uses wireless-domain knowledge to reason about the current algorithm, identify a component worth questioning, and propose an intervention. Workers implement the intervention, while the task evaluator compares the resulting candidate with its parent under matched conditions. Only candidates that achieve a strictly better score are accepted; an LLM-generated rationale alone can never justify a change. After discovery, the selected algorithm is frozen and evaluated on unseen seeds without invoking an LLM.
Mathematical analysis remains complementary to this process. When suitable assumptions yield an analytical characterization, it can explain why a discovered component works and under what conditions. However, whether these assumptions adequately capture the actual channel and whether a component remains beneficial after interacting with the rest of the algorithm often still require empirical validation. Indeed, the poor performance of direct LLM generation in Section 3.2 shows that prior knowledge and reasoning alone are insufficient without feedback from the target environment. BREATHE therefore uses the LLM to bring engineering priors and reasoning into the search, while relying on experiments—not the LLM itself—to validate those hypotheses.