acceptodds
Under review as a conference paper at ICLR 2027

Volterra Flow Matching with Memory for Few-Shot classification

Abstract

While pre-trained vision-language models provide a coarse alignment between image and text embeddings, they often require parameter-efficient fine-tuning (PEFT) to adapt to downstream tasks like few-shot learning. Existing PEFT methods, including prompt tuning, LoRA, and adapter-based approaches, typically perform one-step updates, while recent multi-step methods model refinement via learned velocity fields. However, these approaches typically condition the dynamics on the current state, leaving the potential predictive role of trajectory history underexplored. We hypothesize that, especially in low-data regimes, the instantaneous feature state can be insufficient for predicting the appropriate alignment dynamics. To capture this structure, we propose a Volterra Flow Matching (VFM) framework that introduces a memory variable to encode the history of feature evolution. This memory is constructed via a Volterra integral with an exponentially decaying kernel, leading to a coupled dynamical system that governs both the current state and its accumulated history. Practically, we construct interpolation trajectories between image and text features, and train a lightweight neural operator to predict the drift conditioned on both the instantaneous features and their memory state. During inference, the learned dynamics are integrated using a small number of steps, enabling efficient multi-step refinement. Extensive experiments on standard few-shot benchmarks demonstrate modest yet consistent gains over existing PEFT and Flow Matching Alignment methods, while controlled prediction-risk probes show that the memory state contains additional velocity-predictive information.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Volterra Flow Matching, Memory, Few-Shot classification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aGbfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Volterra Flow Matching (VFM) for few-shot vision-language classification. In contrast to standard flow-matching alignment, where the velocity depends only on the current feature state, VFM introduces a memory variable that summarizes trajectory history through a Volterra integral with an exponentially decaying kernel. The memory can be implemented recursively as an augmented ode, avoiding storage of the full trajectory. The method is evaluated on 11 few-shot classification benchmarks under several settings. Results show generally consistent improvements over CLIP-LoRA and the memoryless FMA baseline, with larger gains in low-shot and difficult-alignment settings. The paper additionally provides synthetic transport experiments, kernel and backbone ablations, and a prediction-risk probe supporting the claim that trajectory history contains information beyond the instantaneous state.

Critical strengths
  1. The Volterra memory admits an efficient recursive ode implementation, and the paper provides theoretical analysis explaining how the augmented state can disambiguate endpoint assignments that are ambiguous from the current state alone.
  2. The authors evaluate across 11 datasets, multiple shot settings, several peft methods and backbones, and controlled synthetic transport tasks. The prediction-risk probe and synthetic experiments are particularly useful because they directly examine the proposed motivation rather than relying only on downstream classification accuracy.
Critical weaknesses
  1. VFM gives the velocity network access to an additional state, but the paper does not include a fully capacity-matched non-history auxiliary-state baseline. The authors themselves acknowledge this limitation. Moreover, GRUFlow achieves comparable performance, suggesting that the main benefit may come from a general history-aware recurrent inductive bias rather than the specific Volterra formulation.
  2. For example, the improvement over FMA decreases as the number of shots increases, and the 1-shot gains are relatively heavy-tailed, with EuroSAT accounting for a large improvement while several datasets show negligible or slightly negative gains. The iterative refinement also adds non-trivial inference latency, which somewhat weakens the peft efficiency motivation.
Questions for authors

see weaknesses

Reviewer_eWwAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In VLM-based few-shot classification, existing flow matching-based approaches for multi-step feature alignment typically rely on memoryless velocity prediction. However, different source–target feature pairs may produce similar intermediate states while requiring different transport directions. To address this issue, this paper proposes Volterra Flow Matching (VFM). VFM introduces a history variable defined by a Volterra integral with an exponentially decaying kernel to construct memory. A velocity prediction network is then trained to predict transport directions conditioned on both the current feature state and its accumulated history. The paper evaluates the proposed method on standard few-shot classification benchmarks, conducts ablation studies, and further validates its effectiveness through several synthetic transport experiments.

Critical strengths
  1. The proposed method is simple and intuitive. The Volterra Memory serves as an additional input to the velocity prediction network, representing historical information with a fixed-size state that can be updated recursively at constant per-step computational cost. This design does not require complex interaction mechanisms or additional dedicated memory networks.

  2. The experimental evaluation is relatively comprehensive. The paper evaluates the proposed method across multiple few-shot settings on 11 datasets. Several synthetic transport experiments are also designed to further validate the effectiveness of the proposed approach.

Critical weaknesses
  1. The methodological novelty is not sufficiently clarified. VFM largely follows the cross-modal feature transport framework established by FMA [R1], while incorporating a structured historical state based on a Volterra integral. The idea of incorporating Volterra integral formulations into Flow Matching has also been explored in related work, such as CaLMFlow [R2]. However, the paper does not sufficiently explain what unique modeling advantages this particular combination offers.

  2. The motivation and effectiveness of the proposed method are not sufficiently explained. Compared with FMA, the key motivation of Volterra memory in VFM is that "different endpoint assignments can produce similar xtx_t while requiring different transport directions." However, the paper lacks a detailed qualitative or quantitative analysis of when and to what extent such trajectory ambiguity occurs in real few-shot classification tasks. Furthermore, the source of VFM's performance gains remains unclear. Although Table 2 shows improvements over FMA, it is not clear whether these gains arise specifically from the proposed Volterra Memory or more generally from additional conditioning information. Since Volterra Memory essentially represents a weighted accumulation of historical feature states, and any intermediate state along the trajectory during training can be constructed from the two Flow Matching endpoints x0x_0, x1x_1, and time, it is worth investigating whether directly conditioning the velocity network on the initial endpoint x0x_0 could achieve similar results. In addition, it remains unclear whether simply accumulating historical states without any kernel would yield comparable performance.

[R1, FMA] Ziqi Jiang et al. Exploring cross-modal flows for few-shot learning. ICLR, 2026.

[R2, CaLMFlow] Sizhuang He et al. Calmflow: Volterra flow matching using causal language models. arXiv preprint arXiv:2410.05292 (2024).

Questions for authors
  1. What is the key novelty of VFM compared with FMA and CaLMFlow? The authors should provide a more detailed discussion of the methodological connections and differences between VFM and these two closely related works, and clearly identify the unique contributions of the proposed method.

  2. Is Volterra Memory necessary? The paper models historical information using a Volterra integral, but the necessity of this particular formulation remains unclear. According to the ablation results, different memory kernels yield relatively small performance differences. Could simpler conditioning or memory mechanisms, such as directly conditioning the velocity network on the initial endpoint x0x_0 or using a simple unweighted accumulation of historical states, achieve comparable results?

  3. The experimental comparisons and analysis are not sufficiently convincing. From the proposed framework, VFM introduces an additional velocity prediction network on top of existing single-step feature alignment methods to perform multi-step refinement. However, this design also introduces additional computational costs (latency and parameters), including the training cost of the velocity network and the cost of multi-step inference. Tables 1 and 3 mainly report improvements in classification accuracy without providing explicit comparisons under matched computational budgets. Although the appendix reports some timing measurements for VFM, it remains difficult to determine whether the improvements over single-step methods are attributable to the proposed mechanism itself or to the additional training and inference computation. Furthermore, FMA is the most direct and important baseline for VFM. Therefore, direct comparisons with FMA should also be included in Tables 1 and 3, rather than being limited to Table 2.

Reviewer_bYwBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VFM to address the limitations of memoryless flow-based methods in cross-modal alignment for few-shot classification. It introduces a memory variable constructed through a Volterra integral with an exponentially decaying kernel, which enables the velocity field to be conditioned on both the current state and its accumulated history. It constructs interpolation trajectories between image and text features, and trains a lightweight neural network to predict history-aware velocity fields. Extensive experiments on multiple few-shot benchmarks demonstrate consistent improvements over existing PEFT and memoryless flow matching methods.

Critical strengths
  1. This paper identifies a limitation of existing PEFT and multi-step alignment methods, which rely on memoryless dynamics, and highlights the challenge of accurately modeling feature trajectories when the instantaneous state is ambiguous.
  2. This paper proposes VFM, a novel framework that introduces history-dependent, marginally non-Markovian dynamics into feature alignment, allowing the velocity field to incorporate historical trajectory information beyond the current feature state.
  3. Extensive experiments on multiple few-shot classification benchmarks demonstrate the effectiveness of VFM across different datasets and few-shot settings.
Critical weaknesses
  1. Figure 2 seems to be an AI-assisted illustration, where the fonts are blurred and difficult to read. Moreover, the content in this figure is relatively simple, lacking sufficient details to clearly explain the proposed memory mechanism and alignment process.
  2. The performance improvements are relatively modest on easy datasets in Table 2, making it difficult to assess the robustness of the proposed method. Moreover, the main experimental results are reported using only one fixed random seed (seed = 1), without mean ±\pm std across multiple runs. Although Appendix A.11 provides multi-seed results for 16-shot settings, the number and specific values of seeds are not reported, which limits the reproducibility.
  3. The experiments do not isolate the contribution of Volterra history mechanism, without evaluating a parameter-matched non-history auxiliary-state control or an x0x_0-conditioned memoryless baseline. Although Proposition 3 establishes that the Volterra memory is sufficient to recover the endpoint displacement under the stated conditions, it does not establish that such memory is necessary.
  4. The main methodological novelty lies in introducing a historical memory state into the flow matching alignment. While this design is intuitive, the overall contribution seems relatively incremental, as it mainly extends the existing flow matching framework with an additional memory component.
Questions for authors
  1. Could authors provide a clearer and more detailed version of Figure 2 to better illustrate the Volterra memory update and history-aware alignment process?
  2. Could authors clarify the seed values which were used in the multi-seed experiments in Appendix A.11? In addition, could the authors provide mean and standard deviation across multiple runs for the main experimental settings?
  3. Could authors provide additional controlled experiments, particularly a parameter-matched non-history auxiliary-state baseline and an x0x_0-conditioned memoryless baseline, to better isolate the contribution of Volterra history modeling?
  4. Could authors further clarify the methodological novelty of the proposed Volterra memory mechanism compared with existing history-dependent dynamical systems and memory-based flow models?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.