acceptodds
Under review as a conference paper at ICLR 2027

Online Coresets for Conditional Flow Matching: Learnt Support-Size and Transport Transfer

Abstract

Flow matching learns a velocity field that transports a source distribution to a data law. In the conditional setting, endpoint memory determines both interpolation inputs and supervised velocity targets. Retaining full history can preserve obsolete regimes under drift, while a fixed small buffer can erase conditional structure. We formulate endpoint memory as a condition-wise model-order problem and introduce Online Memory Compression and Identification with Maximum Mean Discrepancy (OMCI–MMD). The method approximates each conditional kernel mean with a sparse weighted subset of observed endpoints and selects the first support budget whose fully corrective approximation reaches a predeclared MMD sampling radius. Under stationary conditional independence and bounded kernels, we derive support-size brackets that account for approximation error along the computed path. A recent-window analysis separates sampling error, stale-law bias, candidate availability, and numerical error, with current-law guarantees requiring a valid staleness bound. Under explicit reproducing-kernel Hilbert-space witness assumptions, we establish when endpoint MMD controls Wasserstein, Sinkhorn, and fixed-path flow-matching losses. The selected cardinality is frozen before downstream losses are inspected, permitting subsequent fixed-size refinement. In a controlled two-condition, ten-atom neural-flow study, selected support remains at six atoms for the stationary condition and grows to eight after a programmed shift. The selected endpoint laws improve terminal conditional MMD-squared over undersized and stale-history alternatives, while the full-current reference achieves lower held-out velocity error and terminal MMD-squared. These results connect statistically calibrated endpoint selection to conditional generation and quantify its approximation tradeoff.

Primary area:
generative models
Keywords:
conditional flow matching, online coresets, maximum mean discrepancy, support-size selection, distribution shift, witness transfer

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_T5CjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies finite-sample generation risk problem in streaming conditional flow matching, which is caused by condition-wise mode erasure and stale historical data. To mitigate these generation risks, the authors propose a sampling-calibrated model-order framework via an online, sparse MMD program (OMCI-MMD) to maintain the sparse support set, which systematically reduces memory complexity while preserving statistical resolution. Extensive experiments confirm that it significantly mitigates generative hallucinations.

Critical strengths

(1) Fluent writing and strong experimental results.

(2) The theoretical proofs are thorough.

Critical weaknesses

(1) Although the proposed OMCI-MMD provides a principled MMD-based formulation for streaming support management, the empirical evaluation lacks essential baseline comparisons. Specifically, the paper only consider "Retain-All" or "Fixed Small" strategy, while other strategies (e.g., Random Sampling). Without these, it remains unclear whether the performance gains stem from the MMD optimization.

(2) While the paper provides solid theoretical guarantees on sample complexity and bounds the drift-aware error, it lacks a comprehensive empirical evaluation of the computational overhead. Specifically, the runtime cost of executing the kernelized Frank-Wolfe and solving the corrective quadratic program (QP) dynamically in a high-throughput streaming environment is not thoroughly benchmarked against standard baselines (e.g., random sampling).

(3) Minor: The windowed OMCI-MMD framework heavily relies on pre-declared hyper-parameters (e.g., window length LL and safety factor κ\kappa), while no sensitivity analysis are provided.

Questions for authors

Please refer to "Critical Weaknesses"

2 comments

(1) Although the proposed OMCI-MMD provides a principled MMD-based formulation for streaming support management, the empirical evaluation lacks essential baseline comparisons. Specifically, the paper only consider "Retain-All" or "Fixed Small" strategy, while other strategies (e.g., Random Sampling). Without these, it remains unclear whether the performance gains stem from the MMD optimization.

Response to Weakness 1: We agree that this comparison belongs in the main text, and the revision adds it (Figure 4; Appendices E.2 and E.2.1; the full tables appear in our response to Reviewer kPvA, Question 2). At the size OMCI–MMD selects, a random subset with optimized weights matches it on the 10-type stream, where a mass-proportional draw of 8 types almost always picks the 8 substantial ones. On a continuous mixture (40 windows), it is 2.8 times worse in MMD (0.245 against 0.087) and 6.6 times worse in Sinkhorn divergence (0.822 against 0.124). An equal-weight reservoir sample is worse still on both.

Whether the gains stem from the MMD optimization therefore has a two-part answer. Against full history and fixed small buffers, the gains come from the per-condition recent target and the selected size, not from the path. At a fixed size, the MMD path outperforms random selection on continuous data, and is comparable to other structured compressors. A new ablation crosses target, size rule and selection (Appendix E.2.1): with the selected size and the current target, the MMD path and a random subset with optimal weights tie (0.040), equal weights are eight times worse, and a size frozen at the change is five times worse. The image and text pilots now include a random-subset arm at OMCI–MMD's size, which reaches a total variation of 0.44 on images and 0.36 on text, against 0.23 and 0.10 for OMCI–MMD (Appendix E.4).

At scale (Section 5, Figure 3; Appendix E.5), the comparison splits by modality. With GPT-2 and SmolLM2-360M fine-tuned on a stream in which each entity gains a new fact at 10% of its mass, the random subset states the new fact in 75.1% and 75.7% of completions, against 96.1% and 99.3% for OMCI–MMD, and is worse in all five paired streams: it omits the fact in 24 of 100 entity-streams, and where it does, accuracy is near zero. There, the guarantee that no resolvable mode is dropped is what matters. On CIFAR-100 with a class-conditional U-Net, the random subset reaches a lower class-law total variation (0.288 against 0.359) and Fréchet distance, at lower recall; both are far better than the window at this training budget (0.647). The paper reports this result as it is.

(2) While the paper provides solid theoretical guarantees on sample complexity and bounds the drift-aware error, it lacks a comprehensive empirical evaluation of the computational overhead. Specifically, the runtime cost of executing the kernelized Frank-Wolfe and solving the corrective quadratic program (QP) dynamically in a high-throughput streaming environment is not thoroughly benchmarked against standard baselines (e.g., random sampling).

Response to Weakness 2: Per arrival, OMCI–MMD only updates a running mean: O(n)O(n) kernel evaluations with an exact kernel, or O(D)O(D) with DD random features. The Frank–Wolfe path and its re-weighting QPs run only at reporting checkpoints, and each step scores the candidate pool and solves a simplex QP whose dimension is at most the number of kept samples. Measured on 2 CPU cores, one complete readout takes 0.42 s for a 500-sample window with 512-dimensional features and up to 50 kept samples (0.33 s bootstrap, 0.09 s path). For 2000 samples it takes 5.0 s, dominated by the exact-kernel bootstrap; the path itself takes 0.16 s, or 0.50 s with up to 100 kept samples. Random sampling costs O(1)O(1) per arrival, so OMCI–MMD is more expensive; its cost is amortized over the checkpoint interval and runs in parallel across conditions. Appendix E.3 (Table 11) reports these costs and the per-arrival costs (about 1 µs for a reservoir update), next to the generator's training time (6.8 ms per update in our controlled study), and Section 4.3 states the headline figure.

(3) Minor: The windowed OMCI-MMD framework heavily relies on pre-declared hyper-parameters (e.g., window length and safety factor), while no sensitivity analysis are provided.

Response to Weakness 3: The revision adds both sweeps, with noise levels bootstrapped from the window and no simulator (Appendix E.3).

Safety factor κ\kappa (L=500L=500, t=4000t=4000, 60 streams). For κ\kappa from 1.0 to 1.5, OMCI–MMD keeps 8 samples in 60 of 60 streams, with median current-distribution MMD 0.040 (full window: 0.037). At κ=2.0\kappa=2.0 it keeps 7 in 54 of 60 streams, and the error rises to 0.130. The plateau exists because the error curve falls by an order of magnitude between 7 and 8 samples, so any radius in that gap yields the same size.

Window length LL (κ=1.05\kappa=1.05; change at t=2200t=2200; median MMD to the current distribution over 60 streams, median kept samples in parentheses; Figure 9 and Table 10):

LLt=2400t=2400t=2700t=2700t=3200t=3200t=4000t=4000
1000.164 (7)0.159 (7)0.158 (7)0.162 (7)
2500.069 (8)0.054 (8)0.059 (8)0.061 (8)
5000.137 (8)0.038 (8)0.044 (8)0.042 (8)
10000.203 (7)0.114 (8)0.031 (8)0.032 (8)
20000.213 (7)0.165 (8)0.111 (8)0.033 (8)

Full history has error 0.199, 0.178, 0.150 and 0.120 at the four times. LL trades noise against reaction time: very short windows are too noisy to resolve the new modes, and long windows are more accurate once stationary but stay stale for up to LL steps after a change. The selected size alone does not reveal staleness (it reads 8 in stale windows too), which is why the revision adds a data-driven window length. On 60 further streams it matches the best fixed LL once the change is absorbed, at 0.041, 0.030 and 0.033 for t=2700t=2700, 3200 and 4000, but does not react faster than L=500L=500 just after the change.

Questions

Please refer to "Critical Weaknesses"

Response: Please see our responses above. We would like to express sincere gratitude to the reviewer for the positive assessment and for three concrete requests, each of which is now in the revised PDF. We would like to further comment that the relevance to generative modeling for natural language, in terms of rare fact response accuracy tradeoffs with respect to aggregated perplexity should make plain the impact of these results for flow-based generative models. Thank you for your time and consideration.

Reviewer_6gRNOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper deals with flow matching. I am unable to provide further details -- cf. weaknesses below.

Critical strengths

N/A

Critical weaknesses

The paper has one fatal flaw: it makes no visible effort at conveying its contributions in an accessible manner. As an expert in generative models, I am unable to understand the paper, starting from the abstract. Writing is obscure, using convoluted and non-defined expressions. The research problem is not explicitly defined nor motivated. Equations are not explained. The results and their interpretations are unclear and their understanding is left to the reader. Consequently, it is impossible to reliably assess the paper's claims.

Questions for authors

Given the above weakness, this paper is not ready for publication. I would argue it is not even ready for submission. As a result, I strongly recommend rejecting this paper. I have no questions for the authors; I would advise them to submit at a later conference only after a complete revision of the paper.

4 comments

Weaknesses:

  • The paper has one fatal flaw: it makes no visible effort at conveying its contributions in an accessible manner. As an expert in generative models, I am unable to understand the paper, starting from the abstract. Writing is obscure, using convoluted and non-defined expressions.

Response to Weakness 1: We thank the reviewer for stating this plainly, and we agree. The submission was written in the vocabulary of Monte Carlo compression rather than that of generative modeling, which was a blind spot in the way we chose to construct the technical narrative. The revision retains every core technical result and proof and reframes the interpretations around them, while adding numerous concrete experimental results that make plain the impact for generative modeling using flow matching. The abstract now opens with the observation that most conditions come with a handful of examples, states the three ways a training memory fails, and describes the method's two steps, retention and consolidation, in plain words, spelling out maximum mean discrepancy (MMD) where it first appears. The revised Section 2 now begins with illuminating how a flow-matching generator learns from its memory, what MMD measures, and why MMD has a sampling noise level. Monte Carlo terms (model order, kernel herding, coreset) are translated where they first appear, and more broadly, every symbol and technical term is introduced at first use in plain words, with the reference that establishes it. We would appreciate if the reviewer could give the revised paper a quick skim to see if it is more intelligible in their view.

  • The research problem is not explicitly defined nor motivated.

Response to Weakness 2: The revised introduction states the motivation in its first paragraph. A conditional generator is judged one condition at a time, and most conditions come with only a handful of examples. At a condition with ncn_c examples, the expected squared MMD between the empirical and the true distribution is σ2/nc\sigma^2/n_c, whatever the size of the dataset. A flow-matching generator trained to convergence reproduces its training memory, so at such a condition it either replays the few samples or averages them with dissimilar ones, and the averaging erases the rare condition; Figure 1 draws this argument in three panels. In a stream, full history blends regimes, a fixed buffer drops modes that are current but rare, and a condition-blind buffer can match the pooled distribution while assigning every condition the wrong samples (Proposition 1). Section 2.1 then defines the problem: the memory must be current (no obsolete regimes), complete (no mode the data resolve is dropped) and compact (no more samples than the noise level supports). Example 1 works these three requirements through a stream in which a rare type becomes common, where the oracle model order is known exactly (Figure 2). We connect this explicitly factual errors in response generation for rare facts in language modeling in our revised experiments.

  • Equations are not explained.

Response to Weakness 3: Each main result is now followed by what it means for the generator, with appropriate caveats. Theorem 1: the selected memory is never smaller than the best memory at the same resolution, and compression comes from the structure of the distribution. Proposition 2: each term of the drift bound corresponds to one memory rule of Section 1, so full history inflates the staleness term and a short window the sampling term. Proposition 3: a generator trained on the memory is nearly as good on current data as one trained on the full window; conversely, the bound depends on the memory's MMD rather than on the training loss, so no optimizer recovers what the memory omits. Algorithms 1 and 2 give the method as pseudocode, and the worked crossing of Section 3.1 gives the numbers behind one decision: five atoms leave an MMD of 0.1665, and six reduce it to 0.0260, below the radius 0.0629.

  • The results and their interpretations are unclear and their understanding is left to the reader. Consequently, it is impossible to reliably assess the paper's claims.

Response to Weakness 4: Section 5 is now organized as four questions, and each paragraph answers its question in its first sentence: whether retention finds the order (Q1), whether a generator trained on it matches one trained on the full window (Q2), where the gain comes from at matched size (Q3), and what changes in image and text generators trained end to end (Q4). Every figure caption opens with its conclusion in boldface and then states the setup; Figure 3, for example, opens with "OMCI–MMD keeps a rare new fact from 59 times fewer documents than the window; on images, a random subset does better." In every results table, boldface marks the best memory at OMCI–MMD's size or smaller, larger memories are gray references, and OMCI–MMD's rows are shaded, so a win at equal storage can be read at a glance. Table 1 (Appendix A.1) sets our guarantees beside those of kernel Monte Carlo and generative-modeling methods, by setting, size rule and rate, so that the technical contribution can be read off in one place.

Questions

Given the above weakness, this paper is not ready for publication. I would argue it is not even ready for submission. As a result, I strongly recommend rejecting this paper. I have no questions for the authors; I would advise them to submit at a later conference only after a complete revision of the paper.

Response: We have carried out that complete revision, and the revised PDF is available at the link above. We would be grateful if the reviewer could read its first three pages and let us know whether the problem and the contribution are now clear.

I would like to thank the authors for responding to my review and acknowledging the issue I raised. Glancing at the new version of the paper, the presentation seems to be improved, although the manipulated notions remain exposed in a convoluted manner.

Nonetheless, the authors and the committee will understand that a complete revision of the paper is not in the scope of an ICLR rebuttal. Therefore, I maintain my strong recommendation to reject the paper.

Can the reviewer please clarify the meaning of "manipulated notions remain exposed in a convoluted manner" as well as check whether the phenomena of rare fact erasure being observed alongside convergence in aggregate perplexity that we have experimentally showcased, brings the point into clearer view?

Reviewer_kPvAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OMCI–MMD for online endpoint selection in conditional flow matching. It adaptively selects weighted subset sizes using sampling-calibrated MMD thresholds and handles distribution drift with sliding windows. The analysis provides conditional support-size bounds and conditions for transferring MMD guarantees to transport discrepancies and flow-matching losses. Synthetic experiments and small image and text pilots illustrate support-size adaptation and the effects of stale memory.

Critical strengths
  1. Useful problem formulation. The paper highlights how endpoint memory can lose conditional structure or retain outdated distributions. This identifies an important source of generative error that downstream model fitting alone cannot resolve.

  2. Principled selection criterion. OMCI–MMD links support size to statistical resolution through an MMD threshold. The analysis explicitly states the conditions for support-size and downstream error guarantees, clarifying the method’s scope.

Critical weaknesses
  1. Limited experimental scope. The main generative experiments use ten 2D endpoints, with similarly small image and text pilots. Effectiveness on realistic, high-dimensional tasks remains unverified.
  2. Insufficient baseline comparisons. Competitive baselines are discussed, but their quantitative results are not clearly presented. The main comparisons do not establish an advantage over established memory-selection methods.
  3. Unclear practical calibration. Experimental thresholds rely on known simulated distributions. How to obtain reliable thresholds and the additional error bounds from unknown real-world streams is insufficiently demonstrated.
  4. Unproven resource savings. Fewer representative points do not necessarily reduce total memory or runtime. Reported training times are nearly identical and exclude sample-selection costs.
  5. Limited reproducibility. The paper states that a supplementary source archive is provided, but no such archive was available in the materials I reviewed. This limits independent verification of the implementation and reported results.
Questions for authors
  1. How would you calibrate the threshold and obtain the required error bounds from an unknown data stream, without the simulator used in the experiments?
  2. Could you point me to the numerical comparisons with the baselines described in Appendix E? I could not find these results. Were memory and compute budgets matched?
  3. Have you tested the method on larger or more realistic datasets? The current pilots leave me unsure how well it scales beyond small-support examples.
  4. Does the method save memory or time once window storage and coreset selection are included? The reported training times alone do not show a practical benefit.
7 comments

Weaknesses:

  • Limited experimental scope. The main generative experiments use ten 2D endpoints, with similarly small image and text pilots. Effectiveness on realistic, high-dimensional tasks remains unverified.

Response to Weakness 1: We chose small controlled streams because the oracle model order is computable there, which allowed us to test the size rule itself and not only its downstream effect. We agree that this left scale open, and the revision adds a study of 175 trained fits: five backbones, seven memories and five paired streams (Section 5, Figure 3; Appendix E.5). In each of 20 conditions, a new mode enters at 10% of the mass after a change: a held-out fine class in each CIFAR-100 superclass, and a new town for each of 20 fictional entities whose documents pair a fact sentence with a TinyStories story. Every memory receives the same training updates.

  • Text. GPT-2 and SmolLM2-360M fine-tuned on OMCI–MMD's memory, 12.7 documents per entity against 746.7 in the window, state the new town in 96.1% and 99.3% of sampled completions, against 94.7% and 99.1% for the window. A random subset of the same size with optimized weights reaches 75.1% and 75.7%, and is worse in all five paired streams; the most recent items reach 74.9% and 78.6%; full history reaches 5.9% and 27.5%, and states the obsolete town 62.8% and 69.1% of the time. The random subset omits the fact in 24 of 100 entity-streams: where it keeps the fact, GPT-2 states it 98.8% of the time, and where it omits it, 0.1%. No weighting recovers an omitted mode, which is the failure the lower bracket of Theorem 1 rules out (Figure 3b).
  • Images. On CIFAR-100, a U-Net trained on OMCI–MMD's 46.9 images per superclass lowers the class-law total variation from 0.647 for the window to 0.359 and covers 98.5% of the current classes, but a random subset with optimized weights reaches 0.288, at lower recall. At this training budget the image generators do not reproduce their memories (OMCI–MMD's memory is closer to the current class law than the random subset's), so the image comparison measures optimization as much as memory. We report this result as it is, in the abstract and the Discussion.
  • Costs. Compressing to 12.7 documents raises the perplexity of held-out documents for every memory of that size, OMCI–MMD included (from 5.33 to 30.27 for GPT-2), because the kernel resolves facts rather than stories (Remark 1). When the reference window straddles the change, OMCI–MMD keeps the window's obsolete mass (4.9% against 5.1%): removing stale data is the window's task, which is the staleness term of Proposition 2.
  • Insufficient baseline comparisons. Competitive baselines are discussed, but their quantitative results are not clearly presented. The main comparisons do not establish an advantage over established memory-selection methods.

Response to Weakness 2: We agree. The comparisons were run for the submission, but Appendix E only described them. The revision reports them in Figure 4 and Appendix E.2 (Figure 6, Table 7), reproduced in our Response to Question 2 below, adds a crossed ablation of target, size rule and selection (Appendix E.2.1, Figure 7, Table 8), and adds random-subset and most-recent arms of the same size to both end-to-end pilots (Appendix E.4, Figure 10, Tables 13–14). At a fixed size, structured compressors (herding, kk-medoids, coreset anchors) come close to OMCI–MMD, and equal-weight buffers are far worse; the large differences come from which distribution is compressed and how many samples are kept. We have narrowed the claim accordingly: the contribution is a calibrated size rule for a per-condition, recent-window memory, a consolidation step, and guarantees that reach the generator's risk, and any of these compressors can run at the size we select.

  • Unclear practical calibration. Experimental thresholds rely on known simulated distributions. How to obtain reliable thresholds and the additional error bounds from unknown real-world streams is insufficiently demonstrated.

Response to Weakness 3: The guarantees never require a simulator; the experiments used one only to know the true noise level for verification. The revision makes a simulator-free calibration the default, namely the bootstrap noise level computed from the window alone (Section 3.2, Algorithm 2; Appendix D.2, Table 2), and adds a data-driven window length for the staleness term (Appendix E.3). The details are in our Response to Question 1.

  • Unproven resource savings. Fewer representative points do not necessarily reduce total memory or runtime. Reported training times are nearly identical and exclude sample-selection costs.

Response to Weakness 4: The reviewer is right, and we have removed the implication from the paper. In the reported experiments, every current-window policy reads the same 500 observations and every network trains for the same number of updates, so equal training time is by design. "Positive atoms" measures the size of the stored training set, not bytes, and the implementation also keeps the window to compute its target. The submission therefore shows no net memory saving against a 500-observation window. What the method does save, and how the revision shows it, is given in our Response to Question 4.

  • Limited reproducibility. The paper states that a supplementary source archive is provided, but no such archive was available in the materials I reviewed. This limits independent verification of the implementation and reported results.

Response to Weakness 5: We apologize: the archive was not reachable. It is now available at as an upload for the supplementary material. It contains the runners, frozen memory laws, seeds, checkpoints, generated samples and raw metrics of the submitted experiments; a script for every result added in the revision; and the launcher, executed configuration, unit tests and per-fit results of the scale study. The trainer of the 56,354-parameter network of Appendix E.1, which the submitted renderer imported from an internal folder, will be added before the end of the discussion period. Every table and figure of the revision is regenerated as follows, from the archive's root folder (Python 3.10 or later; the controlled experiments run on a CPU in about 30 minutes):

pip install -r experiments/requirements.txt pandas
# controlled experiments and pilots
(cd experiments_rerun/exp1 && python run_continuous_1d_transport_transfer.py --seeds 40 --output-dir rerun_40)
(cd experiments_rerun/exp1 && python run_revision_checks.py && python run_bounded_memory.py)
(cd experiments_rerun/pseudo_inputs && python pseudo_input_proto.py 40 && python gmm_patch.py . 40)
(cd experiments_rerun && python pilots_e2e/run_e2e.py validate && python pilots_e2e/run_e2e.py image && python pilots_e2e/run_e2e.py text)
(cd experiments_rerun && python pilots_e2e/run_consolidation.py --bw-mult 1.0)
# figures and tables, then the PDF
python experiments_figures/appendix_figures.py paper/assets/figures
python experiments_figures/matched_size_table.py paper/tex/generated/matched_size_table.tex
python experiments_scale/analysis/paper_tables.py experiments_scale/results/curated/tables paper/tex/generated
python experiments_scale/analysis/paper_figures.py experiments_scale/results/curated/tables paper/assets/figures/scale
(cd paper && sh scripts/build.sh)

The scale study itself needs GPUs (pip install -r experiments_scale/requirements.txt). Inside experiments_scale, python -m pytest tests and bash smoke/run_smoke.sh check every stage on a CPU in a few minutes, and slurm/RESTRICTED_36H.md gives the steps that launch the executed configuration (slurm/restricted_36h.json) for all 175 fits. The archive's README lists the command behind each individual table and figure. These runs were executed on HPC, routed to about 16 GPUs overnight.

Questions

How would you calibrate the threshold and obtain the required error bounds from an unknown data stream, without the simulator used in the experiments?

Response to Question 1: There are three simulator-free routes, and the revision makes the third the default.

  1. Closed form. The concentration radius of Section 2 holds for any bounded kernel, but it is conservative: 0.243 on our stream against a true 95% level of 0.059. It therefore under-selects (5 samples instead of 6 or 8).
  2. Plug-in mean. The expected squared MMD between a sample of size nn and its source is (E k(Y,Y)−E k(Y,Y′))/n(\mathbb{E}\,k(Y,Y)-\mathbb{E}\,k(Y,Y'))/n, which the window estimates directly with a U-statistic.
  3. Bootstrap (new default). Resample nn points with replacement from the current window, compute their MMD to the window, and take the 95% quantile. On 300 independent windows of our stream, this gives a median of 0.0601 before the change and 0.0587 after (10–90% range within ±0.001\pm 0.001), against simulator values of 0.0601 and 0.0587. It covers the true error in 96.7% and 94.3% of windows (nominal 95%), and selects the same size as the simulator in 300 of 300 windows (Appendix D.2, Table 2).

The drift term is different: a single window cannot reveal its own staleness without assumptions. The revision therefore chooses the window length from the data by a Lepski-type rule, as in ADWIN: keep the longest window whose mean embedding agrees with that of every shorter window within their bootstrap radii. On 60 streams this rule matches the best fixed length once a change is absorbed: at t=3200t=3200 and 40004000 its median MMD is 0.030 and 0.033, against 0.040 and 0.041 for L=500L=500 on the same streams; right after the change, it reacts no faster than L=500L=500. We present it as a heuristic, since the guarantees are stated for a fixed LL. The search-pool and solver errors in the bound are computed by the algorithm and reported.

Could you point me to the numerical comparisons with the baselines described in Appendix E? I could not find these results. Were memory and compute budgets matched?

Response to Question 2: These are now in Figure 4 and Appendix E.2 (Figure 6, Table 7), reproduced below. Every compressor stores the same number of samples as OMCI–MMD and is built from the same current window; the remaining rows are references (full window and full history) and undersized memories. Errors are distances to the true current distribution (lower is better).

Controlled stream after the change: 10 sample types in 2D, 40 independent streams, medians.

MemoryKept samplesMMDW1W_1Sinkhorn (ε=0.5\varepsilon=0.5)
Full current window5000.0380.0330.00092
OMCI–MMD80.0400.0360.00100
Random subset, optimized weights80.0400.0360.00100
Kernel herding, equal weights80.0390.0340.00040
kk-medoids80.0410.0370.00101
Reservoir sample, equal weights80.3210.2740.0700
Same path stopped early60.1940.1820.0278
Static pre-change coreset60.2360.1790.0139
Full history40000.1200.0870.00355

Continuous four-component Gaussian mixture: 40 independent windows, medians (the submission reported a 5-stream pilot).

MemoryKept samplesMMDW1W_1Sinkhorn (ε=0.7\varepsilon=0.7)
Full current window3200.0490.1560.059
OMCI–MMD60.0870.3590.124
kk-medoids60.1100.3530.139
CCVFM entropic anchors60.1130.3620.124
Random subset, optimized weights60.2450.7280.822
Reservoir sample, equal weights60.3661.1862.371
Same path stopped early40.1200.3710.103

On the 10-type stream, the random ties OMCI–MMD because a mass-proportional draw of 8 of the 10 types almost always picks the 8 substantial ones; on the mixture, where no small subset of observed samples is exhaustive, it is 2.8 times worse in MMD. For mixtures, the MMD crossing does not minimize the Sinkhorn divergence: the path stopped two atoms early has a lower median (0.103 against 0.124), although its MMD and W1W_1 are higher, and kk-medoids is marginally better in W1W_1 (0.353 against 0.359). The crossed ablation of Appendix E.2.1 ranks the choices: with the current target, the selected order divides the error of a size frozen at the change by five; with the selected order, the current target divides the error of full history by three; and equal weights multiply the error by eight. Moving the atoms and giving each a learned spread (Gaussian pseudo-inputs, Appendix E.2.2) brings six atoms to the full window's error on the mixture: MMD 0.049 and W1W_1 0.160, against 0.049 and 0.157.

On budgets, stored samples are matched exactly; compute is not, since random and reservoir sampling cost O(1)O(1) per arrival. Selection cost is given in our Response to Question 4.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Grateful to the reviewers for their time and attention. We have substantially strengthened the experimental showcase of conditional bias.

Summary of changes to the paper:

  • In response to reviewer 6gRN, we rewrote the abstract, introduction and preliminaries in the vocabulary of flow matching rather than that of Monte Carlo compression. The paper now opens with why more data does not fix a rare condition (Figure 1), states the problem as three requirements on the training memory, namely that it be current, complete and compact (Section 2.1), defines every symbol and term at first use with the reference that establishes it, follows each result with what it means for the generator, and organizes the experiments as four questions whose answers lead each paragraph and caption. Every result now appears as a figure, in the main text or the appendix, beside the table that gives its numbers.
  • As requested by reviewers kPvA and T5Cj, every baseline is now reported numerically at the memory size OMCI–MMD selects, including random subsets with optimized weights (Figure 4; Appendix E.2), together with a crossed ablation of target, size rule and selection (Appendix E.2.1).
  • As requested by reviewer kPvA, we added a scale study of 175 trained fits with GPT-2, SmolLM2-360M, BERT-tiny, a U-Net and a diffusion transformer on CIFAR-100 and a drifting fact stream (Section 5, Figure 3; Appendix E.5). The noise level is now estimated by bootstrap from the window, without a simulator (Section 3.2; Appendix D.2). We withdrew the implied memory and time savings against a full window, and replaced them with a bounded-memory variant and timed selection (Appendices D.3 and E.3).
  • As requested by reviewer T5Cj, we added sweeps over the safety factor κ\kappa and the window length LL, and a data-driven window length (Appendix E.3).

The rest of the raised issues are addressable, and specified in reviewer-specific responses. We are very grateful for the time expended to allow us to strengthen the submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.