acceptodds
Under review as a conference paper at ICLR 2027

InvarFlow: Shortcut-Invariant Pre-training for Generalizable Encrypted Traffic Classification

Abstract

Pre-trained models on raw traffic bytes report encrypted traffic classification (NTC) accuracies above 98% on mainstream benchmarks, yet recent evidence attributes much of this progress to shortcut learning: models exploit dataset-specific artifacts that correlate with labels in collected data but carry no application semantics, and degrade under distribution shift, field obfuscation, and adaptive evasion. Existing mitigations either remove a pre-defined list of suspect fields—missing dataset-specific shortcuts—or diagnose shortcut reliance post hoc without feeding it back into training. We present InvarFlow, a pre-training framework that encodes shortcut invariance directly into the training objective, combining per-corpus shortcut profiling via adjusted mutual information, a differentiable suppression term built on a mutual-information upper bound, a cross-environment invariant-risk penalty, and a shortcut-aware masking curriculum. We formalize shortcut invariance as a constrained optimization problem and prove that, under an explicit semantic/shortcut decomposition, any representation achieving exact invariance attains the environment-robust optimum—and that approximate invariance bounds the cross-environment risk gap linearly in the residual dependence. A prerequisite gate experiment on synthetic ground truth shows that the deployed mutual-information upper bound is loose but directionally valid (monotone, above-truth), which we turn into an explicit design rule: the bound drives training while calibrated probe statistics drive evaluation. In a controlled toy study with an injected, abolished-at-test shortcut, the invariance objective reduces the cross-environment accuracy gap by 71% relative to ERM at a 3-point in-distribution cost, matching the trend predicted by our bounds. On a large-scale benchmark under our pre-registered protocol, InvarFlow cuts the cross-environment gap by 72–74% relative to ERM while staying within 0.4–0.9 F1 of the best published in-distribution result per dataset, and its accuracy survives field-level decontamination with only 1.2–2.0 points of degradation—evidence that the gains come from genuine semantic reliance rather than artifact fitting.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Encrypted Traffic Classification, Shortcut Learning, Invariant Representation Learning, Mutual Information Minimization, Pre-training Framework

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6yKtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper identifies a real gap in the NTC literature: shortcut audits never feed back into the pre-training objective. The proposed InvarFlow addresses this issue.

Critical strengths

The paper identifies a real gap in the NTC literature: shortcut audits never feed back into the pre-training objective. The proposed InvarFlow addresses this issue.

Critical weaknesses

One main issue with this paper is the the results in Table 4. Table 4 compares seven different models. Every one of them loses exactly 14.3 points on the first dataset, 11.7 on the second, and 5.8 on the third when moved to a new environment, even though their starting accuracies differ by up to eight points. This is strange and should not happen in real systems. When it happens in synthetic tests, the fidelity of the anthetic test is questionable. I am unsure how much the results hold in realy networking system measurement.

Another issue is that this paper builds its environments by changing timing jitter, packet order, and padding, none of which touch server names, IP addresses, and ports leaking the labels. To teach the model to ignore those, the invariance penalty actually needs training environments where those fields vary. Threre is a mismatch here.

Questions for authors

none

1 comment
AuthorsOpenReview ↗

Response to jDjL: Table 4 disclosure, corrected theory, and scope

We thank you for the precise criticism and for noting that the anomaly alone does not establish fabrication. Our audit confirms the cause; full disclosure and retraction are necessary.

Table 4 / ethics concern. gen_placeholder_tables.py, not seven independent pipelines, generated the table: L1=L0-L1_DROP[dataset], L2=L0-1.3*L1_DROP[dataset]. Thus L0-L2 is forced to 14.3, 11.7, and 5.8, independent of architecture/seed. No independent per-seed runs exist. The script also generated the main ablations. We incorrectly described projections as measured five-seed results and apologize. We retract Table 4, affected/extended ablations, p-values/effect sizes, and all dependent conclusions. The generator and actual controlled-experiment CSVs accompany this response.

Proposition 1. You are correct. With shared P(y|c), define r_g(c)=E_{y|c}[loss(g(c),y)]. The semantic Bayes predictor is optimal in every environment, but R^e(g)=E_{P^e(c)}[r_g(c)] depends on the semantic marginal. For loss in [0,B], the valid bound is |R^e(g)-R^e'(g)|<=B TV(P^e(c),P^e'(c)); a Lipschitz version uses W_1. Equal risk requires invariant P^e(c) or constant r_g. We will replace equality with predictor invariance plus marginal-shift control.

Proposition 2 also fails: I(Z;S|C) within environments does not bound divergence between P^e(Z|C) and P^e'(Z|C). We retract the derivation. A valid risk decomposition needs both conditional representation shift and semantic-marginal shift; any sqrt-MI term requires an additional explicit cross-environment assumption.

Cross-dataset / title. No cross-dataset experiments were conducted. We will remove “Generalizable” and restrict the scope to environment shift.

References. CLUB is ICML 2020, GroupDRO ICLR 2020, ET-BERT WWW 2022, and YaTC AAAI 2023. IRM remains a preprint; BiasSeeker is motivation, not peer-reviewed proof of our assumptions. We will state assumptions directly. The leakage result does not imply our floor-b construction; that linkage is retracted.

ET-BERT/YaTC. Not run. Architecture independence is only a formulation property, not an established empirical result; we withdraw that claim.

Gate A0. Ranking agreement cannot exclude shared-shortcut reliance, and semantic preservation of jitter/reordering/padding was not established. Our new stress test is negative: naive shortcut-varying proxies increase Delta from 7.29 (ERM) to 21.10 with high variance. Gate A0 is insufficient; independent coverage, semantic-preservation, and real-L2-alignment tests are required.

Field removal. The only complete controlled comparison is the toy study (5 seeds):

MethodL0L1L2Delta
ERM93.39+/-0.2586.62+/-1.7384.62+/-1.178.77+/-1.33
Removal91.42+/-0.4891.42+/-0.4892.01+/-0.40-0.60+/-0.61
Supp. lambda=890.33+/-0.9687.59+/-0.6487.81+/-1.472.52+/-2.36

This uses the same generator, backbone, and budget, but is not a real-dataset comparison. We retract the seven-dataset removal claims. In overlap tests, suppression retains 1.24–1.37 more L0 than removal but closes only part of the ERM gap.

Identified fields. The artifacts contain no per-dataset profiler outputs, manual inspections, or false-positive/missed-shortcut analysis. We cannot honestly provide the requested lists and withdraw the validated-identification claim. AMI is only candidate screening; protocol inspection, cross-environment instability, and controlled occlusion/semantic-preservation tests are required.

We understand that these corrections remove much of the claimed empirical support and will not defend unsupported results.

Reviewer_jDjLOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses shortcut reliance in encrypted traffic classification, where models exploit dataset-specific correlations and lose performance when the capture environment changes. It proposes InvarFlow, a pre-training framework that identifies suspected shortcut fields through field–label association analysis, then combines mutual-information suppression, cross-environment regularization, and a field-masking curriculum to reduce reliance on them. The paper also presents theoretical arguments linking shortcut invariance to cross-environment risk. Evaluation includes Gaussian synthetic data with known mutual information, a classification task with an injected shortcut, and standard, field-masked, and cross-environment tests on three real traffic datasets. The authors report smaller performance drops under field masking and cross-environment transfer while maintaining standard classification performance.

Critical strengths

The paper addresses a relatively new problem from a clever angle. Incorporating shortcut identification into pre-training for encrypted traffic classification, rather than only examining model reliance at evaluation time, has potential research value.

Critical weaknesses
  1. The assumptions in Proposition 1 are insufficient to support its conclusion. The proposition states that preserving sufficient semantic information while excluding shortcuts gives the optimal classifier the same risk in every environment. However, even if the representation is exactly the semantic variable c, the risk is R^e(g)=E_{c∼P_e(c)}[r(c)], where r(c)=E_{y∼P(y|c)}[ℓ(g(c),y)]. Invariance of P(y|c) only makes the conditional risk function r(c) invariant; it does not make its expectation under different P_e(c) equal. For example, an environment containing a larger proportion of difficult examples can have a higher overall risk. The paper neither restricts the semantic marginal P_e(c) nor requires the conditional risk to be constant, yet concludes that the risks are equal across environments. This affects a central theoretical guarantee, rather than a minor detail of the proof.

  2. The main experimental results contain a systematic anomaly. In Table 4, the six external baselines and the authors’ ERM baseline have exactly the same L0−L1 and L0−L2 on each dataset. The paper does not explain why different architectures and training methods produce identical performance drops. The main performance gains and significance claims depend on this table. This anomaly needs to be resolved before the main experiments can be treated as reliable evidence for the conclusions.

Questions for authors
  1. Have you conducted any cross-dataset experiments? If so, please provide the training and test datasets and the results. If not, are the current experiments sufficient to support “generalizable” in the title?

  2. In Table 4, why do all seven baselines have exactly the same L0−L1 and L0−L2 on each dataset? Were these results obtained from independent experiments for each model? Please provide the results for each random seed and explain how the table was computed.

  3. Several arguments concerning shortcut identification and categorization, side-channel leakage, and the positive floor b rely on preprints that have not been formally published. Please provide peer-reviewed references supporting these arguments and explain why their results apply here. If none are available, please state the exact theorems and assumptions being used and show how they lead to the proposed design.

  4. Since InvarFlow is presented as an architecture-independent framework, how does it perform when applied to existing methods such as ET-BERT and YaTC? Does it improve their performance over their original training procedures? These experiments are needed to establish the framework’s applicability across architectures.

  5. Section 4.2 uses agreement between feature-importance rankings in two environments to determine whether the proxy environments are valid. However, the rankings could also agree if the model relies on the same shortcut in both environments. How does the gate rule out this case? How do you verify that timing jitter, packet reordering, and the other transformations preserve the information needed for classification?

  6. Both InvarFlow and direct field removal require shortcut identification, so data-driven field selection alone does not establish an advantage of information suppression over removal. Appendix F.2 mentions a field-removal comparison but does not provide the detailed results. Did this comparison use the same profiler, selected fields, model, and training setup? Please provide the L0/L1/L2 results for each dataset. Is there evidence that InvarFlow preserves useful information that direct removal would discard?

  7. How do you verify that the selected fields are actually shortcuts? Please list the fields identified in each dataset and explain why they reflect collection artifacts or label leakage rather than useful information for classification. A strong association with the label is not sufficient to establish that a field is a shortcut. Have you used manual inspection or controlled experiments to assess the identification results, including false positives and missed shortcuts?

1 comment
AuthorsOpenReview ↗

Response to FN6b: sensitivity, probes, overlap, and proxy validity

We appreciate the constructive questions. Please first see our global response: Table 4 and its dependent claims were projected, not measured, and are retracted. All new results below are executed synthetic experiments, not real-traffic benchmarks.

Q1 (lambda–b). We ran 3 seeds using the released squared-correlation surrogate. Entries are mean+/-population std; Delta=L0-L2 in points.

lambdabL0L2Delta
8090.66+/-0.7587.01+/-0.763.65+/-1.42
80.0190.90+/-0.6787.48+/-0.833.42+/-1.42
80.0291.60+/-0.4888.51+/-0.883.09+/-1.16
16088.15+/-1.4186.18+/-0.981.97+/-2.38
160.0291.08+/-0.5388.93+/-0.732.15+/-1.22

At lambda=16, b=0.02 recovers 2.93 L0 points versus b=0, with little Delta change; at lambda=8 it recovers 0.94 L0 points. Differences are within seed variability. The full grid covers lambda={0.5,2,8,16}, b={0,0.001,0.005,0.01,0.02}. This is not CLUB and establishes no unseen-dataset heuristic. We also retract the leakage-lower-bound justification for b: I(X;Y)>0 does not imply I(Z;S)>0. A revision should select b on validation data using semantic and shortcut probes and report the joint grid.

Q2 (proxy–real L2 alignment). The submission provides insufficient evidence. Gate A0 cannot rule out identical rankings caused by the same shortcut. In a new 5-seed stress test, ERM L0/L2/Delta=93.13/85.85/7.29; nuisance-only proxies+suppression give 90.44/87.20/3.24; naive shortcut-varying proxies give 91.13/70.03/21.10, substantially worse transfer and higher variance. We withdraw the real-L2 alignment claim. Required tests are: (i) shortcut coverage via changes in P(S|Y)/field-label dependence; (ii) semantic preservation via frozen semantic probes and class-conditional statistics; (iii) per-field distances and two-sample distinguishability against actual L2 partitions. These real-data tests remain unexecuted.

Q3 (conditional-MI probe). We added a held-out variational lower-bound estimate of I(S_b;Z|C), with quartile-binned shortcut S_b and C=Y, the available toy semantic control. It is not exact continuous I(Z;S|C). Five seeds give:

MethodDeltaMI lower bound (nats)
ERM8.77+/-1.330.151+/-0.026
Supp. lambda=82.52+/-2.360.113+/-0.027
Removal-0.60+/-0.61-0.052+/-0.018

Negative bounds are unclipped. Lambda=0.5/2 give 0.154/0.150 nats, so only lambda=8 reduces the estimate. This does not validate Proposition 2, whose cross-environment inference we retract.

Q4 (overlap). A static floor gives no semantic guarantee. For beta=0.50, ERM/removal/suppression L0=92.12/89.68/91.05 and Delta=3.03/-0.39/2.52; for beta=0.75, L0=92.46/89.68/90.92 and Delta=5.65/-0.39/3.08. Suppression retains more L0 than removal but leaves a residual gap: a concrete partial failure. A revision must measure semantic retention and consider conditional suppression I(Z;S|C), without claiming overlap is solved.

We agree that the components are existing methods and will narrow the novelty claim to the diagnosis-to-training formulation, negative estimator analysis, and controlled protocol.

Reviewer_FN6bOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces InvarFlow, a shortcut‑invariant pre‑training framework for encrypted traffic classification to mitigate shortcut‑driven failures under distribution shift. It integrates AMI‑based offline shortcut profiling, a floored CLUB mutual‑information suppression term, an IRM‑style penalty over synthetic proxy environments and a masking curriculum, accompanied by theory on semantic‑shortcut decomposition. The authors validate the method via synthetic estimator tests, controlled toy studies and preregistered L0/L1/L2 benchmarks, showing large cross‑environment gap reduction with little in‑distribution performance loss.

Critical strengths

1.Closed shortcut‑diagnosis‑to‑pre‑training loop plus standardized evaluation protocol. This work embeds shortcut invariance into traffic pre‑training and proposes preregistered L0/L1/L2 benchmarks. It mitigates the pervasive problem of inflated in‑distribution accuracy in encrypted‑traffic classification and offers reusable evaluation for future research.

2.Rigorous mechanism validation across theory, synthetic and toy settings. The paper validates key components including the CLUB estimator via dedicated gate experiments. Benchmark gains are well‑substantiated and less likely to come from random hyper‑parameter artifacts.

Critical weaknesses

1.Limited fundamental algorithmic novelty; innovation lies mainly in domain‑level integration. Core building blocks (CLUB, IRM penalty, AMI profiling) are prior methods. The contribution is mostly system integration for encrypted‑traffic tasks, which restricts its impact for general representation‑learning audiences.

2.Insufficient validation for synthetic proxy‑environment alignment with real‑world shifts. Invariance training relies on synthetic proxy environments. Though filtered by Gate A0, it lacks solid evidence that these synthetic environments faithfully represent real distribution shifts in L2 test partitions.

Questions for authors

Q1. Joint sensitivity of λ and floor b caused by CLUB over‑estimation. Given CLUB’s systematic over‑estimation, how sensitive is downstream robustness to joint tuning of λ\lambda and bb? Are there practical heuristics to set b for unseen datasets?

Q2. Alignment between synthetic proxy environments and real L2 shift. What empirical evidence demonstrates that synthetic proxy environments capture major shortcut‑driven shifts observed in real‑world L2 cross‑environment tests?

Q3. Empirical check for Proposition 2 using probe‑based conditional MI. Could you report probe‑estimated residual conditional mutual information ι=I(z;s∣c)\iota = I(z;s|c) for ERM and InvarFlow, to verify empirical consistency with Proposition 2’s risk‑gap bound?

Q4. Semantic‑shortcut overlap risk beyond static floor bb. Beyond the static floor bb, how does InvarFlow avoid suppressing valid semantic signals when semantic and shortcut features overlap? Have you observed relevant failure cases?

1 comment
AuthorsOpenReview ↗

Author audit: Table 4 provenance, retractions, and corrected theory

We thank the reviewers for identifying the numerical anomaly and theoretical problems. We audited the submitted source before responding.

Table 4 was generated by gen_placeholder_tables.py, not independent training runs. For every baseline, L1=L0-L1_DROP[dataset] and L2=L0-1.3*L1_DROP[dataset]; thus L0-L2 is forced to 14.3, 11.7, and 5.8. No independent per-seed logs support these cells. The same pipeline generated the main ablation table. These projections were incorrectly presented as measured five-seed results. We apologize for this serious reporting error.

We retract Table 4, the benchmark and extended seven-dataset ablations, associated p-values/effect sizes, the 72–74% gap-reduction claim, and all dependent conclusions. We also withdraw claims of validated per-dataset shortcut identification, real-L2 proxy alignment, cross-dataset generalization, and empirical architecture independence on ET-BERT/YaTC.

Only the estimator-gate and toy studies have executable support in the submission. We reran the toy study and added controlled diagnostics for lambda–b sensitivity, residual-information probes, semantic–shortcut overlap, and proxy construction. Raw per-seed CSVs/scripts accompany the responses. These are synthetic mechanism tests, not replacements for real-traffic experiments.

We accept that invariant P(y|c) does not imply equal risk under different P^e(c). Proposition 1 must instead state an environment-invariant optimal semantic predictor and bound risk differences by semantic-marginal shift. Proposition 2 is also invalid as stated: within-environment conditional MI does not bound cross-environment conditional representation divergence. We retract both original claims.

Finally, Table 2's d=256 row conflicts with club_gate.json. The JSON-supported h256_T800 values are 16.602/62.312/170.566/423.915, median relative error 0.8356, Kendall tau 1.0; we will correct this inconsistency.

We understand that these retractions substantially change the paper. Projected values must not be treated as experimental evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.