acceptodds
Under review as a conference paper at ICLR 2027

SimCMTC: A Similarity Comparison-Driven Method for Mobile App Encrypted Traffic Classification

Abstract

Classification of encrypted mobile-app traffic faces three intertwined challenges in open, dynamic environments: label contamination from third-party SDKs, traffic drift caused by app updates, and open-set classes that training cannot cover. We propose SimCMTC, a similarity comparison-driven framework that reformulates classification as a “retrieval + comparison analysis” task. Its core is CMTE, a general embedding model for encrypted mobile-app traffic that learns reusable TCP session representations through large-scale self-supervised pretraining. For unknown traffic, SimCMTC retrieves the most similar labeled samples from a reference vector store and jointly considers vector similarity and nDPI metadata similarity to yield interpretable, open-set-aware predictions. Across five datasets spanning the mobile Internet of China, Iran, and Iraq, CMTE achieves the best closed-set accuracy and F1, and attains Acc/F1 of 0.910/0.898 on TENAPP under fine-tuning-free Top-1 retrieval, approaching the strongest supervised baseline.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Encrypted traffic classification, Traffic embedding, Self-supervised learning, Contrastive learning, Open-set recognition

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rc5xOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SimCMTC aims to identify mobile apps from their network traffic. Its CMTE model learns a compact representation of each TCP connection from packet headers and a picture of packet sizes and timing. Pretraining does not use app labels. The paper tests three uses: adding a classifier and training it on five datasets; finding the closest labeled examples without further model training; and using an agent to compare retrieved examples with network details. The reported results are strongest for classification after further training and retrieval on TENAPP. The full agent is tested only on 1,635 TENAPP cases where the closest-example prediction was wrong. All these cases belong to known apps.

Critical strengths

1 Useful traffic representations. After learning without app labels, CMTE can be trained further for app classification. It reports the highest accuracy and F1 among the methods shown on all five datasets, including tests across different network environments. This gives the work practical value, although the model-selection procedure still needs clarification.

2 Useful retrieval without further model training. On TENAPP, simply using the label of the closest stored example reaches accuracy 0.910 and F1 0.898. This supports a useful alternative to retraining a classifier. It still requires a large labeled reference collection, and the weaker results on other datasets limit how broadly this finding can be applied.

Critical weaknesses

1 The test does not establish recognition of new apps. The experiment used to support unknown-app recognition contains no genuinely unknown apps. Appendix F states that all 1,635 cases belong to the ten known apps. Of these, 1,158 receive the correct app label, giving 70.83%; 271 are incorrectly rejected as unknown. The appendix clearly acknowledges this limitation. These results show that the system can correct some earlier errors, but they do not measure detection of new apps. The agent was also tested only on cases already known to be wrong, so its effect on the full test set is not established.

A reasonable explanation is that the system is meant to refuse a decision when a connection lacks enough app-specific information. That can be useful, but it is different from detecting a new app. To keep the broader claim, the paper needs a test with apps excluded from the known reference classes and a practical rule for when to call the agent. Otherwise, the conclusion should be limited to error correction and cautious refusal.

  1. The final test may have been used to select the model Sections 4.1 and 4.2 say that the best test result during training was reported. If the final test set was used to choose the model, the reported scores may be optimistic. Applying the same rule to all methods does not remove this problem. The wording may instead refer to a separate validation set used to choose the model, but Appendix C does not make that split clear.

Please identify the data used to choose the model and any separate final test. If an independent final test already exists, this concern may be resolved by the split and training records. If not, the main comparison needs an independent evaluation before the reported gains can be relied on.

Questions for authors

1 Data used to choose the model Does “best test result” mean a separate validation set, or the final test set? Please give the split and the rule used to choose the saved model. A separate final test would substantially reduce the second concern above.

2 What an unknown result means Is the target the app that started the connection, the service it contacted, or a refusal to decide? Is there an existing test containing genuinely new app classes? If so, please report both missed known apps and new apps incorrectly accepted as known. If not, the unknown-app claim needs a narrower wording or a direct test.

3 What the agent adds Is there a full-test result using a rule that decides when to call the agent without knowing the true label? Please also compare with a simple method using the same retrieved examples and network information. The agent model, full instructions, decision rules, retries and running cost would help make this comparison checkable.

4 Benefit of contrastive training Table 2 gives slightly better matching results after removing MoCo. Which measured task shows the benefit claimed for keeping it? Also, does the vector analysis use the TENAPP training set or test set? A direct result using the existing models would be more informative than the number of dimensions alone.

1 comment

Response to Reviewer rc5x

We thank the reviewer for the careful reading. All changes are in the revised paper; we answer each sub-question separately.


1. Data used to choose the model

1.1 Is "best test result" a validation set or the final test set? We acknowledge the earlier wording was ambiguous and could be read as selecting the checkpoint with the best test score, which is optimistic. The revised main results (Table 1) report the final-epoch test result under a fixed schedule: CMTE, ET-Bert, and TrafficFormer are fine-tuned for 10 epochs and YaTC for 200 epochs (§4.1). No test-set feedback selects any model.

1.2 What is the split and the saved-model rule? The rule is deterministic and applied identically to all methods (final-epoch checkpoint). Except CHNAPP — the only same-source 9:1 split (train 2,684 / test 295, App. C) — every test set is collected in a different network environment or time from its training set (CrossNet2021: env. A to B; Blend-60: 5 device/network scenarios; MingledApp: Tehran/Qom to Iraq; TENAPP: different dates). Hence final-epoch results still reflect genuine out-of-domain generalization. The splits and the checkpoint rule are reported in §4.1 / App. C.


2. Meaning of the unknown result and genuinely new apps

2.1 What does "unknown" mean — the initiating app, the service, or a rejection decision? The target label is the hosting app class of the session. "Unknown" is a deliberate rejection decision: when neither the domain/DPI evidence nor the retrieval-fusion can confidently attribute a session, it is reported as one of the open-set categories (SDK / OAS / unknown) rather than forced into a target app (§3.3). This is the cautious, abstaining behavior the reviewer describes.

2.2 Is there a test with genuinely new app classes? Yes. §4.2.2 holds Bilibili, Rednote, and Kuaishou out of the reference store as open-set classes. On the full 18,250-session TENAPP test set, the framework reaches 0.8866/0.8833 vs. the pure-DPI agent's 0.8116/0.7817 (Table 2). App. F resolves these flows into SDK (951), OAS (4,226), and unknown (1,467) sessions — held-out/third-party traffic is safely rejected, not accepted as a target class.


3. What the agent adds

3.1 Is there a full-test result under a label-free "when to call the agent" rule? Yes. The invocation rule is label-free: only L1-unresolved sessions enter the agent/L2 path (§3.3). Running the complete framework end-to-end on the full test set gives 0.8866/0.8833 vs. the pure-DPI baseline 0.8116/0.7817 (Table 2).

3.2 What does the agent add over a simple method using the same retrieved examples and network information? Two things, and one controlled comparison. The agent (i) avoids hand-writing a large complex rule base: it autonomously judges the ownership of each extracted domain (TLS SNI / HTTP Host / nDPI SNI) as a target app, SDK, other app, or unknown, so new/unseen domains need no new rules; and (ii) makes autonomous open-set decisions — attributing unfamiliar domains to SDK/OAS/unknown and falling back to retrieval-fusion when a domain cannot be attributed, instead of forcing a class (§3.3). For the controlled comparison, w_vec=0 in App. G scores the same embedding-retrieved candidates by nDPI similarity only: best 0.726/0.585 vs. best full 0.7758/0.6779 (validation unknown sessions), i.e., +5.0% Acc / +9.3% Macro-F1 from the semantic-similarity term; pure-DPI without retrieval is 0.1707/0.0365. Retrieval itself relies on CMTE embeddings.

3.3 Agent model, full instructions, decision rules, retry, and running cost. The complete SimilarityDPI skill (pipeline, mandatory-condition gate, fusion config w_vec=0.60, fuse_min=0.75) is public at https://anonymous.4open.science/r/SimCMTC-ED76/skills/SEVENAPPForSessionPcap_en/SKILL.md; the agent model, full instructions, decision rules, retry mechanism, and running cost are described in Appendix F.


4. Benefit of contrastive training

4.1 On which measurement is the claimed benefit realized? On the embedding-space geometry that retrieval depends on, not a single-point accuracy gain. §4.4 / Table 4 measures it directly: adding debiased contrastive learning lowers the average inter-sample similarity and increases the effective dimensionality, so the space is well spread and avoids dimensional collapse.

4.2 Is the vector/PCA analysis on the training or test set? On the TENAPP train-set embeddings (and the NUDT-Mobile-mini subset for the threshold analysis), computed from the existing model (§4.4, App. D): inter-sample similarity 0.4641$\to$0.0798 and PCA dims 77$\to$209 on TENAPP train (mini: 0.2628$\to$0.0011, 189$\to$461).

Reviewer_iBVROpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SimCMTC, a similarity comparison-driven framework for encrypted mobile application traffic classification. Its core component, CMTE, learns TCP session embeddings through multimodal self-supervised pretraining. Built on ModernBERT, CMTE integrates packet-header representations with visual features derived from packet sizes and timing information, leveraging masked language modeling (MLM), RetroMAE-style reconstruction, and debiased MoCo contrastive learning. During inference, SimCMTC retrieves similar labeled traffic samples and employs an agent equipped with the SimilarityDPI skill to compare embedding similarities and nDPI metadata for traffic classification and interpretable predictions. The framework aims to address challenges such as label contamination, traffic drift, and open-set recognition.

Experiments on five datasets from China, Iran, and Iraq show that fine-tuned CMTE outperforms the evaluated classification baselines. Without fine-tuning, CMTE achieves 0.910 accuracy and 0.898 macro-F1 on TENAPP using Top-1 retrieval. Additional ablation studies and an agent-based analysis of 1,635 Top-1 retrieval failures are provided.

Critical strengths
  1. A practically motivated framework combining multimodal representation learning with retrieval-based classification: The paper presents a practically motivated design that reformulates encrypted traffic classification as a retrieval-and-comparison task rather than relying solely on a conventional classification head. This design provides a potentially flexible alternative to conventional closed-set classifiers, as the reference database can be updated without necessarily retraining the embedding model.

  2. Promising empirical performance across diverse network environments: The proposed CMTE achieves the consistently strong classification performance among the evaluated baselines across five datasets covering different geographical and network environments. The additional ablation studies and embedding-space analyses also provide useful insights into the contributions of individual components.

Critical weaknesses
  1. Limited methodological novelty and insufficient justification of key design choices: While the overall framework is practically motivated, its core components largely build upon existing techniques, and the specific methodological contributions beyond their integration are not sufficiently established. In particular, the underlying rationale of the debiased MoCo mechanism is not adequately explained. Although Figure 1 illustrates the overall workflow of the SimilarityDPI skill, the specific decision-making mechanism of the agent remains insufficiently explained, particularly how embedding similarity and nDPI metadata are combined to produce final predictions and identify unknown classes. The presentation of the methodology further weakens its clarity. Figures 1 and 2 suffer from small fonts and insufficient visual hierarchy, making the core components difficult to follow. Moreover, important methodological details are deferred to the appendix, while some secondary content, such as the tokenizer efficiency analysis in Section 3.2.2, would be more appropriate in the experimental section. The authors should reorganize these sections to better emphasize the central technical contributions.

  2. Insufficient experimental coverage and lack of in-depth analysis: The experimental evaluation is limited relative to the scope of the paper's claims. Beyond the main comparisons and ablation studies, dedicated evaluations of the three motivating challenges—open-set recognition, traffic drift, and label contamination—are largely missing. The experimental analysis also lacks sufficient depth. Section 4.4 and Appendices D–F are overly report-oriented, containing extensive visualizations, statistics, and detailed case reports without consistently extracting meaningful insights or investigating the underlying mechanisms. For example, the PCA analysis does not adequately explain why improved embedding dimensionality does not translate into better Top-1 retrieval performance. The authors should prioritize hypothesis-driven experiments and mechanistic explanations over descriptive reporting, while streamlining the supplementary analyses and highlighting the most informative findings in the main text.

Questions for authors
  1. Debiased MoCo mechanism What is the justification for using a fixed similarity threshold to identify potential false negatives, and how can these samples be distinguished from informative hard negatives? Furthermore, Table 2 shows that removing MoCo improves Top-1 retrieval performance. How do the authors reconcile this result with the claimed benefits of contrastive learning, beyond the observed improvement in embedding dimensionality?

  2. Validation of the central claims The paper identifies open-set recognition, traffic drift, and label contamination as three major challenges, but lacks dedicated experiments to evaluate them. In particular, Appendix F contains no genuinely unknown classes. Can the authors provide direct experimental evidence demonstrating that SimCMTC effectively addresses these three challenges?

  3. Effectiveness of the complete framework The agent-based evaluation is conducted only on samples where Top-1 retrieval has already failed. What is the end-to-end performance of the complete SimCMTC framework on the full test set? How does it compare with simpler alternatives, such as Top-K voting or embedding similarity combined with nDPI metadata without an agent?

2 comments

Response to Reviewer iBVR

We thank the reviewer for the detailed and constructive feedback. All changes are in the revised paper; pointers below refer to it. We answer each sub-question separately.


1. Novelty and justification of key design choices

1.1 What is the contribution beyond integration? Two aspects. Conceptually, SimCMTC reformulates encrypted-traffic classification as a retrieval + comparison task and decides by fusing semantic embedding similarity with nDPI metadata similarity, fused = w_vec·sim + (1−w_vec)·m (Eq. 4–5, §3.3). To our knowledge, no prior work classifies mobile encrypted traffic this way: CBR provides a retrieval paradigm but does not explore the embedding model, and STAR is adapted only to website fingerprinting (§2.1). Architecturally, CMTE is a single multimodal model that jointly trains MLM, RetroMAE-style sentence reconstruction, and MoCo contrastive learning (Eq. 3, App. B) — a combination not previously used.

1.2 How are embedding similarity and nDPI metadata combined? The Method now specifies this fully (§3.3). It is a deterministic two-stage decision: an L1 domain/DPI attribution, and an L2 fusion of the retrieved Top-K candidates by Eq. 4, where the nDPI similarity m is a fixed, field-wise weighted sum of data-ratio, packet-length/goodput/packet-count, and IAT/packet-length standard-deviation terms (Eq. 5), preceded by a DPI-consistency gate (ja3s, SNI, TLS version, nDPI protocol). The weights (w_vec=0.60, fuse_min=0.75) are fixed hyperparameters tuned on a held-out validation set (§4.2.2, App. G); the L2 fusion contains no LLM, and the agent is confined to L1 attribution and open-set fallback. A full worked execution is in App. E.

1.3 Organization and figures. The tokenizer-efficiency analysis was moved to the experiment appendix (App. A); the core skill/fusion details now live in §3.3, and appendices are limited to training objectives and supporting analyses. Figures 1 and 2 were enlarged with clearer visual hierarchy.


2. Direct evidence for the three motivating challenges (Q2)

2.1 Open-set recognition (new, controlled). We designate Bilibili, Rednote, and Kuaishou as open-set classes outside the seven target apps and remove their traffic from the reference store (§4.2.2). On the full 18,250-session TENAPP test set, the complete framework reaches Acc/Macro-F1 of 0.8866/0.8833, vs. the pure-DPI agent's 0.8116/0.7817 (Table 2). App. F shows these error test samples are resolved into SDK (951), OAS (4,226), and conservative unknown (1,467) sessions, so held-out and third-party traffic is safely rejected rather than forced into a target class. Unlike the earlier failure-subset analysis, this contains genuinely held-out classes.

2.2 Traffic drift. Because classification is retrieval against a refreshable reference store, drift handling requires no re-training: drifted traffic is embedded (inference only) and appended to the store (§3.1, §4.2.1). TENAPP's cross-time collection (Oct 2025–Jan 2026, train/test on different dates) is the concrete illustration (App. C).

2.3 Label contamination (SDK). Self-supervised embeddings do not inherit noisy app labels, and shared-SDK sessions (e.g., Soul heavily embeds getui/zztfly/oceanengine SDKs; 408/951 SDK sessions come from Soul) are isolated as SDK/OAS rather than merged into a host-app class (App. F).

2.4 Depth / mechanism, not only reporting. We condensed the PCA/training-curve/margin analyses and tied them to the design: §4.4 and App. D show that without contrastive regularization the [CLS] vector collapses toward the reconstruction objective, and quantify it with effective dimensionality (Table 4) — explaining the counter-intuitive ablation rather than merely tabulating it.


3. Debiased MoCo (Q1)

3.1 Basis for a fixed threshold. Under our MoCo setup the query is a masked + dropped view while the positive key is the unmasked, no-dropout view of the same sample, so positive pairs have near-1 similarity. A fixed threshold θdeb=0.9\theta_{\mathrm{deb}}=0.9 therefore cleanly separates true negatives from likely pseudo-negatives; the sensitivity analysis over {0.85, 0.9, 0.95} (Table 3) shows 0.9 is the best compromise (0.85 wrongly removes hard negatives; 0.95 admits more pseudo-negatives).

3.2 How are pseudo-negatives distinguished from informative hard negatives? A fixed threshold is a coarse approximation that may occasionally drop an extreme hard negative, but the ablation shows the debiasing benefit dominates. Moreover, the final decision does not rely on the embedding alone: at inference the retrieved candidates are re-ranked by fusing the embedding similarity with the nDPI metadata similarity (Eq. 4–5, §3.3), so any residual ambiguity among closely-similar (hard-negative-like) candidates can be further separated by nDPI evidence. This is quantified in the hyperparameter sweep (App. G, validation unknown sessions): vector-only similarity (w_vec=1) reaches at best 0.748/0.653, versus 0.7758/0.6779 for the best fusion (w_vec=0.60) — i.e., adding nDPI metadata similarity improves over the embedding alone (pure DPI-only, w_vec=0, is 0.726/0.585).

3.3 Reconciling the "removing MoCo helps" result. That observation came from the small NUDT-Mobile-mini subset. We now repeat the key-module ablation on the full NUDT-Mobile (Table 3): full 0.910/0.898, w/o visual encoder 0.863/0.840, w/o MoCo 0.906/0.893 — the mini-subset "improvement" reverses, so it was a small-scale artifact. The value of contrastive learning is in the embedding-space geometry, measured directly (§4.4/Table 4): inter-sample similarity 0.4641$\to$0.0798 and PCA dims 77$\to$209 on the TENAPP train set (mini: 0.2628$\to$0.0011, 189$\to$461). We report the modest accuracy effect honestly and attribute the primary benefit to this embedding-space geometry.


4. End-to-end value of the full framework (Q3)

4.1 End-to-end performance on the full test set. §4.2.2 evaluates the complete skill-driven agent on the full TENAPP test set (not only the failure subset): Acc/Macro-F1 0.8866/0.8833 vs. the pure-DPI agent 0.8116/0.7817 (Table 2), about +7.5% Acc and +10% Macro-F1.

4.2 Comparison with Top-K voting. Top-K voting only aggregates neighbor labels and cannot emit "unknown" when the true class is absent, unlike our joint class+open-set decision (Eq. 4, fuse_min; App. F yields 1,467 unknown + 951 SDK + 4,226 OAS). We also measured Top-K voting on the full test set:

DatasetTop-1Top-3Top-5Top-10
CHNAPP0.68140.68140.67800.6576
CrossNet20210.20370.20370.22120.2286
ITC-Net-Blend-600.45020.44770.44580.4338
ITC-Net-MingledApp-Iraq0.27970.28740.28630.2735
TENAPP0.91040.90690.90430.8994

Top-K is not reliably better than Top-1, and on most datasets it is worse (e.g., TENAPP 0.9104$\to$0.8994 from Top-1 to Top-10; Blend-60 0.4502$\to$0.4338; CHNAPP 0.6814$\to$0.6576). This is expected because the reference store contains repeatedly-sampled (duplicate/near-duplicate) sessions, so a Top-K neighborhood is often dominated by the same class's duplicates; aggregating more neighbors adds little new information and can dilute the correct Top-1. This is why our framework re-ranks with nDPI similarity (Eq. 4–5) instead of naive voting.

4.3 Comparison with an embedding + nDPI combination without the agent. This combination is our L2 pipeline (Eq. 4–5, deterministic, LLM-free); the agent only adds L1 attribution and open-set fallback. To isolate the semantic-similarity term on identical candidates, w_vec=0 in App. G scores the same embedding-retrieved candidates by nDPI similarity only: best 0.726/0.585 vs. best full 0.7758/0.6779 (validation unknown sessions), while pure-DPI without retrieval is 0.1707/0.0365.

Reviewer_cTFkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SimCMTC, a framework that reformulates encrypted mobile traffic classification as "reference vector store construction and retrieval and comparison-based classification" rather than directly training a classifier. At its core is CMTE, a multimodal embedding model built on ModernBERT with a lightweight visual encoder, trained with a joint self-supervised objective combining MLM, RetroMAE-style sentence reconstruction, and debiased MoCo contrastive learning. The authors argue that this design avoids training-label contamination (e.g., SDK sharing) and the closed-set assumption, and enables fine-tuning-free retrieval plus open-set recognition of unknown traffic. Experiments span five datasets from China, Iran, and Iraq, and the complete pipeline is wrapped as a SimilarityDPI skill that lets an agent produce interpretable classification reports by fusing vector similarity and nDPI metadata similarity.

Critical strengths
  • The problem framing is practically motivated and well argued. The discussion of SDK contamination and traffic drift as reasons to avoid supervised closed-set training is convincing, and reformulating classification as retrieval + comparison is a sensible response.
  • The debiased MoCo strategy is simple and effective: masking negatives whose similarity to the anchor exceeds a threshold, without clustering or distributional assumptions. The ablation (Table 2) shows a 0.070 accuracy drop without debiasing, which is a meaningful gain for a lightweight modification.
  • The analysis in Section 4.4 linking the absence of contrastive learning to dimensional collapse is a genuine insight. The PCA evidence (42 vs. 105 vs. 209 dimensions to reach 95% variance) is a clean way to demonstrate that contrastive learning's value lies in embedding-space geometry rather than single-point accuracy, and it honestly explains the counterintuitive ablation result where removing contrastive learning slightly raises Acc/F1.
Critical weaknesses
  • The relationship between the retrieval results and the final agent decision is under-specified. How are vector similarity and nDPI metadata similarity combined? Is there a fixed weighting, or does the agent decide case by case? Without this, the 70.83% figure on the 1635 failed sessions is hard to attribute to the framework versus the underlying LLM agent.
  • The gains over supervised baselines on cross-environment data are modest in absolute terms. On ITC-Net-MingledApp, CMTE (Fine-tune and head) reaches 0.730/0.701 while TrafficFormer is at 0.184/0.216; the improvement is large relatively but the absolute performance is still low, and the paper acknowledges this only briefly.
  • Fine-tuning-free retrieval results are highly sensitive to reference-store scale, as the authors themselves note for CHNAPP and CrossNet2021 (0.204/0.163 and 0.681/0.629). This suggests the method's practicality depends heavily on having a large, well-covered reference set, which weakens the claim that it addresses traffic drift in a fine-tuning-free manner.
Questions for authors

Suggestions:

  • Operationalize interpretability: define what constitutes a good rationale, and either run a small human evaluation or measure consistency between reported rationale and ground-truth nDPI fields.
  • Specify the fusion rule between vector similarity and nDPI metadata similarity, including whether weights are fixed, learned, or agent-determined; a sensitivity analysis would strengthen the 70.83% result.
  • Add a full-data ablation on NUDT-Mobile for at least the debiasing threshold and the contrastive-learning module, so that the ablation conclusions are not tied to the mini subset.
2 comments

Response to Reviewer cTFk

We thank the reviewer for the positive assessment and the constructive suggestions. All changes are in the revised paper; we answer each sub-question separately.


1. Retrieval results vs. the final decision; the fusion rule

1.1 Are vector similarity and nDPI-metadata similarity combined by fixed weighting, learning, or per-case agent decisions? By a fixed, deterministic rule (§3.3): fused = w_vec·sim + (1−w_vec)·m (Eq. 4), where sim is the CMTE cosine similarity and m is a field-wise weighted sum of nDPI components (data ratio, average packet length, goodput, packet counts, and packet-length/IAT standard deviations; Eq. 5), preceded by a DPI-consistency gate (ja3s, SNI, TLS version, nDPI protocol). The weights are fixed hyperparameters tuned on a held-out validation set (w_vec=0.60, fuse_min=0.75; §4.2.2, App. G) — neither learned per case nor chosen by the agent.

1.2 Can the result be attributed to the framework rather than the underlying LLM? Yes. The L2 fusion is a pure, LLM-free function; the agent only performs L1 domain/DPI attribution and open-set fallback (§3.3). The full-test result is therefore attributable to the deterministic retrieval + multi-cue fusion, not to the underlying LLM . Moreover, the whole classification is inference-only over a fixed embedding model — no classifier is trained, and none needs retraining as traffic drifts — a clear operational advantage. A complete, traceable worked example (fields, similarity, fused score, final decision) is in App. E.


2. Modest absolute gains on cross-environment data

2.1 Is the low absolute performance on ITC-Net-MingledApp discussed? Yes, explicitly in §4.2.1. MingledApp is a genuinely hard cross-region setting (train Tehran/Qom, test Iraq).

2.2 What claim do you retain? The consistent relative improvement over all supervised baselines across environments (fine-tuned CMTE: 0.715/0.724 on Blend-60, 0.967/0.958 on TENAPP, Table 1), stated with the low-absolute-performance caveat for MingledApp.


3. Sensitivity of fine-tuning-free retrieval to reference-set scale

3.1 How do you address the dependence on reference-store size (CHNAPP, CrossNet2021)? Honestly (§4.2.1). This is exactly the mechanism we claim: a larger, refreshed store improves retrieval, and TENAPP's cross-time collection (a large reference set) reaches 0.910/0.898 via Top-1 (Table 1). Crucially, updating the store requires no model training — only inference (embedding the drifted traffic and appending it), which is the operational advantage of a retrieval-based method over retraining a classifier; timely store updates are therefore part of the intended deployment, not a hidden assumption.

3.2 Does a single Top-1 miss limit the method? No — retrieval is combined with comparison analysis. The class-wise Top-5 Recall@5 is consistently much higher (0.996 vs. 0.910 on TENAPP, Table 1), and the full-test framework reaches 0.8866/0.8833 (Table 2).


4. Suggestion (a): operationalize interpretability

4.1 How is a "good rationale" defined and measured? The rationale is the explicit, traceable field-wise comparison between the query session and the retrieved candidates (App. E gives a complete example with the exact nDPI fields and fused score). App. F measures the consistency between the reported decisions and the actual nDPI/domain fields — e.g., all 951 SDK outputs are verified against their explicit SNI domains across ~17 vendors (Table F.1).


5. Suggestion (b): fusion weights and sensitivity

5.1 Are the weights fixed, learned, or agent-decided, and is there a sensitivity analysis? Fixed hyperparameters tuned on the held-out validation set (App. G). App. G reports the full grid w_vec ∈ {0,…,1.0} × fuse_min ∈ {0.65,…,0.80} × six DPI-weight configurations (264 combinations). Performance peaks at w_vec=0.60, fuse_min=0.75 (validation Acc/Macro-F1 = 0.776/0.678) and degrades smoothly toward both extremes of w_vec and higher fuse_min, substantiating the stability of the chosen values.


6. Suggestion (c): full-data ablations

6.1 Ablations on the full NUDT-Mobile. Done (Table 3): full 0.910/0.898(Acc/F1), w/o visual encoder 0.863/0.840, w/o MoCo contrastive learning 0.906/0.893. Removing the visual branch clearly hurts; removing MoCo leaves Acc/F1 almost unchanged but collapses the embedding-space geometry (§4.4/Table 4: 0.0798$\to$0.4641 inter-sample similarity, 209$\to$77 dims).

6.2 Debiasing-threshold ablation on full data. We are transparent that the threshold ablation (0.85 / 0.9 / 0.95 vs. none) is on the mini subset (§4.3), since each full pretraining run costs 3 epochs × 4 H100 and 600 GPU hours. However, θdeb=0.9\theta_{\mathrm{deb}}=0.9 was fixed on the independent mini subset and applied unchanged to the full model — never tuned on full training data or any test set — and the full model with this fixed threshold shows the expected behavior (Table 3 core-module ablation, Table 4 geometry).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response Summary to the Area Chair

Response Summary to the Area Chair

SimCMTC — A Similarity Comparison-Driven Method for Mobile App Encrypted Traffic Classification

We thank all three reviewers. The revised paper addresses the key concerns as follows; pointers refer to the revised paper.

1. Reviewer positions

  • iBVR (1, reject): (i) limited novelty / weak justification of design choices; (ii) missing dedicated experiments for the three challenges; (iii) unclear end-to-end value of the agent.
  • rc5x (2, weak reject): (i) model-selection bias; (ii) no genuinely unknown apps; (iii) agent vs. a simple method on the same inputs.
  • cTFk (3, weak accept): (i) fusion rule clarity; (ii) modest absolute cross-environment gains; (iii) reference-size sensitivity; (iv) full-data ablations.

2. How each concern is addressed

2.1 Model selection (rc5x). Main results now report the final-epoch test result under a fixed schedule (CMTE/ET-Bert/TrafficFormer 10 epochs, YaTC 200), applied identically to all methods and using no test-set feedback (§4.1, Table 1).

2.2 The three challenges (all). §4.2.2 holds Bilibili/Rednote/Kuaishou out of the reference store; on the full 18,250-session TENAPP test set the framework reaches 0.8866/0.8833 vs. pure-DPI 0.8116/0.7817 (Table 2), with open-set flows resolved into SDK/OAS/unknown (App. F). Drift needs no retraining (store refresh, §3.1/§4.2.1); SDK contamination is isolated as SDK/OAS (App. F).

2.3 Fusion and the agent's role (iBVR, cTFk). §3.3 specifies a deterministic two-stage decision and fused = w_vec·sim + (1−w_vec)·m (Eq. 4–5) with fixed w_vec=0.60, fuse_min=0.75. The agent only does L1 domain attribution and open-set fallback — no hand-written rule base, autonomous domain judgment, and no LLM in L2. A worked example is in App. E.

2.4 Full-data ablations and geometry (cTFk, iBVR). Full NUDT-Mobile (Table 3): full 0.910/0.898, w/o visual 0.863/0.840, w/o MoCo 0.906/0.893. MoCo's value is the embedding-space geometry (§4.4/Table 4: inter-sample similarity 0.4641$\to$0.0798, PCA dims 77$\to$209 on TENAPP train).

2.5 Agent vs. a simple method, and Top-K voting (iBVR, rc5x). Full-test framework 0.8866/0.8833 vs. pure-DPI 0.8116/0.7817 (Table 2). w_vec=0 in App. G isolates the semantic term on the same embedding-retrieved candidates: best 0.726/0.585 vs. best full 0.7758/0.6779; pure-DPI without retrieval 0.1707/0.0365. Top-K voting cannot emit "unknown" (App. F), and — measured on the full test set — it is not more accurate than Top-1 (e.g., TENAPP 0.9104$\to$0.8994 from Top-1 to Top-10), since the reference store contains repeatedly-sampled duplicates that dominate the neighborhood.

2.6 Interpretability and sensitivity (cTFk). The rationale is the explicit field-wise comparison (App. E); App. F measures consistency with true nDPI/domain fields. The full w_vec × fuse_min × DPI-weight grid is in App. G.

3. Core contribution

Conceptual: reformulating encrypted-traffic classification as retrieval + comparison, deciding by deterministically fusing semantic embedding similarity with nDPI metadata similarity (Eq. 4–5) — to our knowledge not done before for mobile encrypted traffic (§2.1). Architectural support: a single multimodal model jointly trained with MLM + RetroMAE + MoCo (Eq. 3, App. B).

4. Request

Given (i) the corrected model-selection protocol, (ii) full-scale held-out-class evidence for the three challenges, (iii) full-data ablations resolving the counter-intuitive result, and (iv) a clarified, isolated agent contribution, we respectfully request major revision rather than rejection.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.