- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
FlowGuard combines a depth-4 in-switch decision tree with off-switch LLM analysis of traffic-window summaries and drift-triggered tree adaptation. The window stage detects suspicious traffic and identifies victim subnets, which are used to derive weak flow labels; a second LLM decides whether a drift event warrants retraining, and a gate compares candidate and deployed trees on held-out weak labels and original labeled data. Offline experiments on streams composed from CIC-DDoS2019 and IoT-23 report window-level F1 of 0.968 on the development evaluation stream and 0.894 on a frozen validation stream. Adaptation improves the deployed tree on the former stream, while separate measurements characterize victim localization, packet-drop rates, and LLM inference cost.
Critical strengths
-
Separating immediate packet-path classification from slower aggregate analysis is a sensible systems design. The paper details switch-compatible features, tree compilation, weak-label construction, and gated updates, and provides an accessible implementation. This makes the proposed closed loop more concrete than simply applying an LLM to individual traffic records.
-
The evaluation includes useful controls and candid limitations: a same-digest rule, a simple volume baseline, a frozen validation stream, chronological test-then-train replay, and retraining-policy/sensitivity studies. Reporting actual packet-drop fractions separately from window alarms is particularly valuable, as is documenting that frequent retraining can degrade performance even with largely correct weak labels.
Critical weaknesses
-
The incremental value of the LLM stages is not yet convincing relative to simpler alternatives. The detection F1 is 0.968 versus 0.961 for the fixed volume threshold; an evaluation-tuned volume threshold achieves 0.995 there and 0.889 on validation, versus 0.894 for the LLM (Section 4.2). Packet-volume victim ranking already contains the target in 99.8% of explained windows and has higher exact-/24 accuracy than the LLM (Table 5). Retraining on every drift matches the LLM-triggered tree/pipeline scores on both streams, with the LLM saving one training attempt per stream (Table 8). These results show some efficiency and worst-segment benefits, but do not establish a substantial learning advantage over a well-controlled lightweight detector and adaptation policy on the same information. The large headline gains over static flow/packet classifiers conflate model choice with access to aggregate temporal information.
-
The deployment evaluation omits key information and timing constraints. Accuracy uses all-packet offline digests, whereas the switch samples approximately 1/1024 of packets; the effect of sampling/sketch error on detection, weak labels, and adaptation is unmeasured. Full-digest detection takes 1.47 s per approximately 1.07 s window, and end-to-end queueing, update delay, and packet mitigation are not evaluated. The minimal digest is a useful alternative, but combined detection/explanation/adaptation serving still requires measurement under actual switch-exported traffic.
-
The evaluated traffic is too narrow to establish robustness for the intended DDoS setting. Each attack segment has one source and one or two targets, leaving distributed sources, subnet-wide victims, and several digest components insufficiently tested. The frozen validation stream is valuable, but its 30.3% window false-alarm rate also raises questions about operational robustness across changing benign traffic. Broader independent attack captures and benign shifts are needed before the systems claims are convincing. Together with the limited demonstrated LLM advantage, these gaps motivate weak rejection.
Questions for authors
-
Can the authors compare with a lightweight learned or adaptive window-level detector using the same digest, and isolate the benefit of LLM retraining decisions under matched weak-label and retraining budgets? Uncertainty across independent attack captures would help interpret the small validation gap.
-
Can the authors report results using actual switch-exported sampled digests under distributed, high-churn traffic, including switch resources, export bandwidth, queueing latency, and time until effective packet mitigation? The repository README also states that every classified flow exports a flow_digest; please include this adaptation traffic in the overhead accounting.
-
How does the safety gate protect against systematic victim-prefix labeling errors or benign changes outside the original labeled data? Since periodic updates still degrade performance despite the shared gate, an evaluation of accepted-update failures and a genuinely held-out labeled audit would clarify what safety the gate actually provides.
4 comments
Reply to Reviewer vath [Part 2/2]
W3. Traffic coverage and validation false alarms
On the validation stream, the full-digest LLM had a lower false-alarm rate than the fixed volume threshold, the threshold rule, and SISTAR. The full-digest LLM flagged 30.3% of the 211 DDoS-negative validation windows, compared with 39.3% for both the threshold rule and the volume threshold and 60.2% for SISTAR (Table 4). On four further DDoS-negative IoT-23 traces (16,918 windows), the original full-digest LLM, without recalibration, flagged 0.007% of windows on benign Somfy-01 and Somfy-04 traffic (4.02% for the fixed volume threshold), 0.09% on 3-1 (3.25%), and 7.3% under heavy scanning in 49-1, where the full-digest threshold rule flagged every window.
For attacks on many hosts, we built a multi-victim replay from capture 60-1. All tested logistic regression and GBDT variants scored 0% recall on this replay. The full-digest LLM flagged the first attack window and reached 9.6% recall overall. Independently captured distributed attacks are the next step for this part of the evaluation.
Manuscript changes. Section 4.4 and Appendix H.
Q3. What the safety gate protects
For each update, the gate limits the loss of balanced accuracy on the original labeled data to one percentage point relative to the deployed tree and requires improvement on held-out weak labels.
To inject systematic errors, we replaced each eligible victim prefix with the most frequent alternative /24 before generating both the fitting and gate labels. Over five seeds of LLM-triggered retraining on evaluation, enforcing the gate under these errors raised overall tree F1 from 0.451 to 0.547, with unchanged pipeline F1. With the original weak labels on validation, it raised pipeline F1 from 0.8885 to 0.8912.
We then scored every attempted update against ground-truth labels on flows from the next four adaptation intervals, excluding the whole retraining candidate pool. With corrupted labels, the gate accepted a candidate at one evaluation drift in three of five seeds even though balanced accuracy on the audited flows fell from 0.793 to 0.454. On validation, it accepted the update in all five seeds, and pipeline F1 stayed at 0.8885 with or without the gate (Appendix I).
Manuscript changes. Section 3.5 and Appendix I.
We hope these additional results address your questions. We will respond promptly to any further questions during the discussion period.
Follow-up on our response
Dear Reviewer vath,
Thank you for the careful and detailed review. In response, we added learned detectors on the same digest (Section 4.3, Table 13), compared retraining triggers under matched budgets (Section 4.2, Appendix I), accounted for the flow_digest export traffic (Appendix D), evaluated four additional attack-free traces (Appendix H), and tested the safety gate under injected labeling errors (Appendix I).
Please let us know if any point needs more detail. If these additions resolve your concerns, we would be grateful if you could re-evaluate the submission. Thank you very much.
Best regards, Authors
Brief note before the discussion closes
Dear Reviewer vath,
We appreciate the care you put into your review. Before the discussion closes today, we would welcome any feedback on our response and the additions in the revised PDF. We are happy to clarify anything that remains unclear.
Thank you very much.
Best regards, Authors
Reply to Reviewer vath [Part 1/2]
We thank the reviewer for the careful reading and the concrete requests. All changes in the revised PDF are highlighted in orange, and the new experiments are collected in Appendices H–J.
Using the same full digest, the LLM achieved higher F1 on both original streams than logistic regression and GBDT trained on CIC-DDoS2019 windows. We trained logistic regression and gradient-boosted trees (GBDT) on CIC-DDoS2019 digests and tested them on the shift to IoT-23. Because all three models receive the same window digest, this comparison separates the choice of model from access to aggregate temporal information. With the full digest on the original evaluation and validation streams, the LLM reached F1 of 0.968 and 0.894, GBDT reached 0.950 and 0.837, and logistic regression reached 0.555 and 0.655.
The gap widened on two further IoT-23 floods, a moderate-rate SYN flood in capture 34-1 and a high-rate UDP flood amid scanning in 43-1. With the same full digest for all methods, the LLM was the only detector to detect all four recorded attack episodes, reaching F1 of 0.927 and 0.865 (mean 0.896) against 0.301 for GBDT and at most 0.500 across all ten non-LLM baselines.
Full-digest logistic regression scored 0% recall on 43-1, while full-digest GBDT scored 0% recall on 34-1 and flagged every window on 43-1 (100% false-alarm rate). Seven of the ten non-LLM baselines reached at most 3.4% recall on 43-1, where heavy background scanning masks the volume surge, and the other three scored 0% recall on 34-1 (Appendix H, Table 13).
Manuscript changes. Sections 4.1, 4.3, and 5, and Appendix H.
Under matched label and attempt budgets on the evaluation stream, the LLM trigger reached a tree F1 of 0.852 against 0.777 for retraining on every drift. We held the initial tree, weak-labeling rule, and safety gate fixed and varied only the trigger. With at most 668 admitted labels and three attempts, both triggers made two attempts. Retraining on every drift spent 398 labels at an early drift in 60-1 and had too few left for the later 35-1 shift. The LLM skipped the 60-1 drift and used 507 labels at 35-1. On validation, both triggers selected the same update under a cap of 793 admitted labels and two attempts.
With exactly 100 labels per attempt, both triggers used 200 labels on evaluation and reached the same tree and pipeline F1. The LLM trigger dropped fewer benign packets (0.648% against 0.736%). The two triggers also tied on validation (Appendix I, Table 16).
Manuscript changes. Section 4.2 and Appendix I.
Line-rate drops come from the compiled tree, which matches the offline tree on every evaluated flow. The compiled switch tree agrees with the offline tree on all training and evaluation-stream flows (Appendix B). Per-flow state uses a fixed table of 216 slots (Appendix B). The tree classifies flows in the data plane, while the window stage runs off-switch, so concurrent inference leaves the switch's export workload unchanged. The switch updates the packet and byte counters and the LogLog sketches on every IPv4 packet. Packet sampling therefore affects only the top-destination share, victim-prefix, and cross-server similarity fields of the full digest, and the minimal digest uses none of them.
On the evaluation stream, an average of 259 flows/s reached classification. Flow records and sampled packet records together need an estimated 0.0455 Mb/s of payload bandwidth on average and 1.49 Mb/s in the peak 1.074 s window (Appendix D).
On one worker, the minimal digest took 0.54 s per request and processed 1.85 windows/s against an arrival rate of 0.93, with validation F1 of 0.886 against 0.894 for the full digest. Detection scores in the paper use offline digests computed from all packets, and the timings cover LLM requests only. End-to-end measurements of queueing and update-to-mitigation delay are left to future work.
Manuscript changes. Section 4.2 and Appendix D.