acceptodds
Under review as a conference paper at ICLR 2027

NetCast: Learning Compact Latent Representations from Network Traces

Abstract

Network foundation models promise reusable representations for diverse traffic analysis tasks, but existing models can produce concentrated embedding spaces and show weak alignment with interpretable traffic statistics, raising questions about whether their representations capture meaningful traffic behavior. We translate these limitations into two concrete design objectives: learning behavioral structure through future burst-state prediction and encouraging representation diversity through distributional regularization. Guided by these objectives, we introduce NetCast, which predicts future burst states from causal flow history using a shared encoder that combines packet content with traffic metadata and a state-distribution regularizer that encourages variation across encoded bursts. We evaluate NetCast across four traffic datasets through centered kernel alignment (CKA) with measured network statistics and intrinsic analyses of embedding geometry. NetCast achieves the highest reported mean CKA of 0.104 and the lowest average pairwise cosine similarity of 0.55 among the compared trained models, while reducing angular concentration on every dataset. A regularizer ablation further supports the contribution of distributional regularization to this geometric improvement. Together, these results demonstrate the potential of burst-state prediction with distributional regularization to produce better-structured network representations that more closely reflect measurable traffic behavior.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
network measurement, network traffic analysis, traffic classification, foundation models, representation learning, latent prediction, self-supervised learning, computer network

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PpRbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper propsoes a network foundation model called NetCast, which is pretained by predicting the latent embeddings of future traffic bursts based past history. 4 datasets were used for validation.

Critical strengths

The idea of pre-training by predicting the latent embeddings of future traffic bursts from hisoty of past bursts has value.

Critical weaknesses

No headline claim is statistically supported. Numbers in tables 1-5 lack statisticaly significancne. Average with standard deviation can be more convincing if put in those tables to make any statistically valid conclusions or claims. The mean CKA margin 0.002 is small. The downstream average margin 0.26 (84.39 vs. 84.13) is also very small that may not support the conclussion of better performance when comapred with existing methods. Appendix D did show the standard devistion but those standard deviations are big (e.g., 0.405 ± 0.369), too big to claim the the average margin is statistially meaningful.

Also, no downstream advantage over the strongest fair baseline. Against netFound-base (225M), NetCast ties on all three tasks. The efficiency difference (48M vs 225M) may not be a meaningful win when it runs on a netowrk device, in terms of the networking metrics such as latency and throughput. Both fits in a modern GPU easily. So, the practical impact on netowrking is very minimum.

Questions for authors

none

1 comment

Rebuttal:

We thank the reviewer for the assessment and for recognizing the value of predicting future burst representations from past traffic. We clarify our contribution in parameter efficiency and the interpretation of Appendix D, particularly the review's reasoning about GPU capacity and standard deviation.

Parameter efficiency and downstream performance

The review assesses downstream benefits mainly through the comparison with netFound base. However, an important contribution is competitive task performance within a smaller parameter budget. The comparison with the closest model in size, netFound small (53M), therefore also deserves consideration.

ModelParametersCrossmarkets Acc@10ISCXVPN F1CIC IDS2017 F1
netFound small53M47.2789.2099.96
netFound base225M60.6691.7599.97
NetCast (ours)48M60.6492.5399.99

At a comparable parameter scale, NetCast improves Crossmarkets Acc@10 from 47.27 to 60.64 and VPN F1 from 89.20 to 92.53. These comparisons use the same pretraining corpus and input processing pipeline.

Compared with the 225M netFound base, NetCast achieves nearly identical Crossmarkets performance and a higher VPN F1 point estimate with approximately 21% of the parameters. Describing the results only through the average downstream score difference of 0.26 overlooks model size. Our central result combines a smaller parameter budget with competitive task performance.

GPU capacity and practical deployment value

The review argues that practical value is minimal because both models fit on a modern GPU. We believe that whether a model fits on a GPU is insufficient to assess parameter efficiency or deployment value.

Fitting on a GPU only establishes that the model meets the device's capacity constraint. It does not imply equal memory consumption, supported batch sizes, inference latency, throughput, or concurrency. These depend on the hardware and workload and cannot be inferred from GPU fit alone.

NetCast (48M) uses approximately 79% fewer parameters than netFound base (225M) while achieving competitive downstream performance. This demonstrated parameter efficiency does not disappear because both models fit on the same GPU. Fewer parameters do not automatically imply lower latency or higher throughput, but sufficient GPU capacity does not establish that resource differences have no practical value either. We therefore ask that the assessment consider model size and task performance rather than GPU fit alone.

Our new measurements on a single NVIDIA A100 provide direct evidence of this distinction. At batch size 256, NetCast (48M) processes approximately 1,958 flows/s, compared with 1,548 for netFound-small (53M) and 330 for netFound-base (225M). It also supports batch size 1,024, whereas netFound-large (640M) runs out of memory at batch sizes 512 and 1,024. These results demonstrate higher batched throughput and greater supported batch capacity in the measured setting. We invite the reviewer to consult Section 4.5 and Figure 3 of the revised manuscript for the full throughput, latency, and memory-capacity comparison.

Appendix D and the main CKA comparison

The review cites 0.405 ± 0.369 from Appendix D to question the average CKA difference in Table 3. However, this standard deviation describes alignment with one traffic feature, not uncertainty in the average difference between models.

A large standard deviation for one feature does not show whether the aggregate model difference is statistically significant. That requires uncertainty estimates for the comparison itself. Appendix D therefore cannot directly support the conclusion that the average CKA difference is not meaningful.

Reviewer_67MjOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the prevalent latent space collapse issue in network foundation models, where traffic embeddings become highly concentrated and show weak alignment with interpretable traffic statistics. It proposes NetCast, a self-supervised pre-training framework that predicts future burst representations from causal flow history, paired with a state-distribution regularizer to counteract the trivial optimization shortcut of representation shrinkage and preserve embedding diversity. With only 48M parameters, NetCast achieves the highest mean CKA with traffic statistics and consistently lower angular concentration across four datasets, and ablation studies verify the regularizer’s direct effect on mitigating collapse, offering a new paradigm focused on intrinsic embedding quality for network representation learning.

Critical strengths
  1. Conceptual insight into latent space collapse: The paper identifies and formalizes that pure prediction objectives create a trivial optimization shortcut: models can shrink representations to reduce prediction loss while erasing meaningful traffic distinctions. This addresses a root cause of poor representation quality in network foundation models, moving the field beyond accuracy-only evaluation.

  2. Rigorous intrinsic evaluation paradigm: It combines CKA alignment with traffic statistics and embedding geometry diagnostics, paired with strictly controlled identical pre-training pipelines against baselines. This directly assesses representation quality rather than relying only on downstream task scores.

Critical weaknesses
  1. Extremely limited experimental scope: All pre-training uses one single campus dataset and only one 48M small model size, with no cross-domain or larger-scale validation. This critically weakens the claim of being a general foundation model approach, as generalizability across traffic types and model scales remains unproven.

  2. Weak and inconsistent semantic alignment evidence: The mean CKA improvement is marginal (0.002 over netFound-small), driven entirely by the MAWI dataset, and lacks statistical significance testing. On three of four datasets, baselines achieve higher CKA. This undermines the core claim that representations better reflect measurable real traffic behavior.

Questions for authors
  1. CKA significance and dataset sensitivity: The 0.002 mean CKA advantage is driven almost entirely by the MAWI dataset. Have you run statistical significance tests on this gap? What is the mean CKA across the other three datasets excluding MAWI? This directly bears on the core claim of improved traffic-statistic alignment.

  2. Model scale scalability: Only a single 48M compact model is evaluated. Have you tested larger model sizes (e.g., comparable to 225M netFound-base)? It is unclear whether latent collapse diminishes with larger capacity, and whether the regularizer still delivers consistent benefits at scale.

  3. Regularization details and alternatives: What are the exact values of key hyperparameters (weight λ, number of random projection directions M)? Have you compared this characteristic-function-based regularizer against simpler distribution constraints (e.g., covariance-to-identity penalty)? This affects both reproducibility and the assessment of novelty.

3 comments
AuthorsOpenReview ↗

Response on Q1 (CKA significance and dataset sensitivity)

Rebuttal:

We thank the reviewer for raising concerns about CKA significance and dataset dependence. We acknowledge these limitations. We would also like to highlight that NetCast achieves higher average CKA on the two evaluated operational network datasets, CAIDA and MAWI.

Dataset scope

We use 46,179 flows from Crossmarkets. For CIC IDS2017, we use the same version as netFound, with 495,718 flows across eight classes, rather than the full public dataset. CAIDA and MAWI contain 1.5 million and 1 million flows, respectively. Both come from real operational networks outside the campus pretraining corpus.

Results on operational networks

Using the values in Table 3, we calculate the equally weighted mean CKA across CAIDA and MAWI:

ModelCAIDA CKAMAWI CKAOperational network mean CKA
netFound small0.0460.0590.0525
netFound base0.0450.0300.0375
NetCast0.0340.1080.0710

NetCast's mean CKA is 35.2% higher than netFound small and 89.3% higher than netFound base on these two datasets. On MAWI, NetCast reaches 0.108, compared with 0.059 for netFound small. This is an absolute gain of 0.049. It also exceeds the best other baseline on MAWI, which scores 0.070, by 0.038.

The overall mean gain of 0.002 does not fully show this difference across datasets. NetCast shows a larger gain on MAWI, a real network dataset with one million flows. We provide this additional comparison to show where the gains occur. We retain the original evaluation across all four datasets and keep their weights unchanged.

Limitations

We acknowledge that MAWI drives this advantage. NetCast has lower CKA than netFound on the other three datasets. We also acknowledge that the current results do not establish statistical significance. A large dataset does not replace uncertainty analysis. Our claim is therefore higher average alignment with traffic statistics on the evaluated operational network subset, rather than consistent or statistically significant gains across all datasets.

AuthorsOpenReview ↗

Response on Q3 (Regularization details and alternatives)

Rebuttal:

Hyperparameters

Both Small and Base configurations use λ=0.09\lambda = 0.09 and M=1024M = 1024. Following the notation in the paper, the total loss is:

LNetCast=Lpred+0.09,Rstate(P^burst).\mathcal{L}_{\mathrm{NetCast}} = \mathcal{L}_{\mathrm{pred}} + 0.09\\,\mathcal{R}_{\mathrm{state}}(\widehat{P}_{\mathrm{burst}}).

The prediction loss is the MSE between predicted representations and future target representations with stopped gradients. The state distribution regularizer acts on both valid context and target burst representations.

At each forward pass, we sample 1024 directions from a standard Gaussian distribution and normalize them to unit vectors. For each direction, we use 17 equally spaced frequency points in [0,3][0,3] to compare the empirical characteristic function with the standard normal reference, e−u2/2e^{-u^2/2}. We will include these settings in the implementation details.

Difference from a covariance penalty

The covariance penalty is: ∥Cov⁡(Z)−I∥F2.\left\lVert \operatorname{Cov}(Z)-I \right\rVert_F^2.

The covariance penalty constrains second order statistics, but does not directly control the mean or the full distribution. Even zero mean and identity covariance do not imply a Gaussian distribution.

Our state distribution regularizer matches characteristic functions along random directions, capturing distributional differences beyond second order statistics. Finite directions and frequency points make this an approximate constraint, rather than a guarantee of Gaussian representations.

Comparison with a covariance penalty

We compare our state distribution regularizer with a covariance penalty that encourages the covariance matrix to approach the identity matrix. Values below are average pairwise cosine similarities (↓\downarrow). Bold indicates the lowest value among configurations in each column. A negative reduction indicates an increase in cosine similarity.

ConfigurationCrossmarketsCIC-IDS2017CAIDAMAWI
NetCast w/o regularization0.540.620.650.66
NetCast w/ covariance-to-identity0.490.560.630.70
NetCast w/ Rstate\mathcal{R}_{\mathrm{state}}0.520.500.530.64
Reduction vs. no regularization0.020.120.120.02
Reduction vs. covariance penalty-0.030.060.100.06

Our state distribution regularizer achieves lower angular concentration than the covariance penalty on three of four datasets, with notable gains on CAIDA and MAWI, both drawn from real operational networks. Cosine similarity decreases from 0.63 to 0.53 on CAIDA and from 0.70 to 0.64 on MAWI, giving a 12.0% reduction in the average across these two datasets.

Although the covariance penalty performs better on Crossmarkets, our regularizer improves over no regularization on all four datasets. In contrast, the covariance penalty increases angular concentration on MAWI, indicating less consistent benefits in this evaluation.

AuthorsOpenReview ↗

Response on Q2 (Model scale scalability)

Rebuttal:

Model scalability

We thank the reviewer for raising this question. We have evaluated NetCast base with 212.83M parameters, closer in scale to the 225M netFound base.

ModelParametersCrossmarketsCIC-IDS2017CAIDAMAWIAverage
netFound small53M0.680.650.790.780.7250
netFound base225M0.630.600.730.770.6825
NetCast small48M0.520.500.530.640.5475
NetCast base212.83M0.550.520.510.560.5350

NetCast base achieves lower angular concentration than netFound base on all four datasets, with an average cosine similarity of 0.5350 versus 0.6825, approximately 21.6% lower, using 14.5% fewer parameters.

Compared with NetCast small, it improves on both operational network datasets: 0.53 to 0.51 on CAIDA and 0.64 to 0.56 on MAWI, with small increases on the other two datasets. These results show that NetCast's geometric advantage extends to a larger scale.

Reviewer_zXGrOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

NetCast learns network traffic representations by predicting future burst embeddings from flow history, with a distributional regularizer to discourage collapse. The model combines packet-derived tokens with traffic metadata. Evaluation covers representation geometry and alignment with traffic statistics on four datasets, plus three classification tasks. The clearest findings are lower angular concentration across datasets and improved downstream performance over the similarly sized netFound-small. The average CKA improvement is small and comes from MAWI.

Critical strengths
  • The comparison with netFound uses the same pretraining corpus and input processing. NetCast substantially improves Crossmarkets and VPN classification over netFound-small at a similar parameter count, making this a useful empirical result.

  • The geometric improvement is consistent across all four datasets, including operational traffic collections. The regularizer ablation also establishes that this component contributes to the reduction in angular concentration.

Critical weaknesses
  • It is unclear why greater alignment with traffic statistics should imply a better representation. CKA measures similarity to the chosen CICFlowMeter features, rather than usefulness by itself. If those features are the reference for a good representation, a natural question is why not use them directly. I would like to see the same statistical features evaluated with a strong classifier, such as gradient-boosted trees, under the same data splits. The existing nPrintML comparison does not directly answer this question. This would help establish what NetCast adds beyond the statistics used to evaluate it.

  • The evaluation does not establish that NetCast preserves changes within a flow. The motivation emphasizes evolving network behavior, but the reported geometric metrics do not distinguish variation across flows from variation within a flow. For example, embeddings could mainly encode stable application or protocol identity: different flows would remain separated, and neighboring bursts would be easy to predict, without capturing changes in the exchange. The paper needs evidence that temporal order or within-flow evolution contributes to the learned representation, beyond these stable attributes.

Questions for authors
  • How does NetCast compare with a classifier trained directly on the CICFlowMeter features used in the CKA evaluation? Does combining those features with NetCast embeddings provide any additional benefit?

  • Does adding the regularizer improve downstream performance, or is its demonstrated benefit limited to cosine geometry?

  • What evidence distinguishes learning within-flow dynamics from encoding stable differences between flows? For example, does disrupting burst order during pretraining materially reduce representation utility?

1 comment

Rebuttal:

We thank the reviewer for the thoughtful assessment and for recognizing the controlled comparison, compact model performance, and consistent geometric improvements. We clarify the scope of our evidence below.

Statistical alignment and representation utility

We agree that CKA alone does not define representation quality. We use CICFlowMeter features as an interpretable reference and assess CKA together with embedding geometry and downstream performance.

We have not evaluated a classifier trained directly on these features or their combination with NetCast embeddings. Since nPrintML uses different inputs, it does not replace this comparison. Our results therefore do not establish superiority over a strong CICFlowMeter feature classifier.

Roles of the predictor and regularizer

The predictor and regularizer serve complementary roles. The predictor encourages representations to retain information useful for anticipating future bursts. The regularizer constrains their distribution and discourages representation shrinkage as a shortcut to reducing prediction error.

The ablation directly supports the regularizer's effect on angular concentration. The downstream results support the utility of the complete framework, including predictive learning. They do not separately quantify the downstream contribution of either component.

Within-flow dynamics

We agree that lower angular concentration does not distinguish within-flow dynamics from stable differences between flows. Causal burst prediction provides a temporal learning signal, but this alone does not prove that the model captures changing network states.

Our current evaluation does not separate within-flow and between-flow variation or include a burst-order ablation. The supported conclusion is that the framework learns useful, less concentrated representations, rather than that its benefits have been specifically attributed to within-flow dynamics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.