Flow Matching for Generative Modeling

Problem

Framing

Continuous normalizing flows train by simulation through ODE solves, which makes large-scale image modeling slow and brittle. The paper replaces simulation-based CNF training with Flow Matching: regress a vector field that generates a chosen probability path, then choose paths that are easier to learn and sample from.

Currently Used Methods

Direct antecedents

• @dinhNVP2017 — exact normalizing flows with tractable Jacobians for likelihood-based generation.
  • Limitation in context: discrete invertible layers restrict path flexibility and architecture choices.
• "Neural Ordinary Differential Equations" — continuous-time flows trained through ODE simulation.
  • Limitation in context: training requires repeated solves, which dominates CNF cost.
• @songScoreSDE2020 — score-based diffusion with continuous-time stochastic dynamics.
  • Limitation in context: strong generation quality, but sampling and likelihood use expensive integration.
• @DenoisingDiffusionProbabilisticModels2020 — diffusion training via denoising objectives.
  • Limitation in context: objective is tied to diffusion paths, not arbitrary transport paths.
• "Score-Based Generative Modeling through Stochastic Differential Equations" — probability-flow ODE view for diffusion models.
  • Limitation in context: inherits diffusion-path constraints rather than optimizing path geometry.

Proposed Method

Architecture

For images, the model uses the U-Net from @dhariwalDiffusionBeatGANs2021 with minimal changes and predicts a time-conditioned CNF vector field $v_t(\mathbf{x})$. The method changes the training target and path, not the backbone.

Loss / Objective

Flow Matching trains the CNF by direct vector-field regression.

The paper then uses the conditional form with Gaussian paths

and conditional vector field

Sampling Rule / Algorithm

Sampling solves the learned CNF ODE from base noise at $t=0$ to data at $t=1$.

Training Procedure

• Optimizer: Adam, $\beta_1=0.9$, $\beta_2=0.999$, weight decay $=0$, $\epsilon=10^{-8}$.
• Precision: FP32 for CIFAR-10 and ImageNet-32; mixed FP16 for ImageNet-64/128/256.
• Likelihood solver: dopri5 with $\mathrm{atol}=\mathrm{rtol}=10^{-5}$.
• Image backbone: Dhariwal-Nichol U-Net.
• Time sampling uses $t \in [0, 1-\epsilon]$ with $\epsilon=10^{-5}$.

Evaluation

Datasets

• CIFAR-10
• ImageNet 32$\times$32
• ImageNet 64$\times$64
• ImageNet 128$\times$128
• ImageNet 256 upsampling

Metrics

• Bits per dimension (NLL/BPD)
• FID
• Inception Score
• NFE
• PSNR
• SSIM

Headline results

• CIFAR-10 unconditional: FM matches diffusion baselines in likelihood and improves low-NFE sampling.
• ImageNet 32$\times$32: FM-OT gives the best speed-quality tradeoff among compared CNF objectives.
• ImageNet 64$\times$64: FM-OT remains competitive in FID while using fewer function evaluations.
• ImageNet 128$\times$128: the paper reports strong likelihood and FID with the same U-Net backbone.
• ImageNet 64$\times$256 upsampling: FM supports conditional generation with reported FID, IS, PSNR, and SSIM.

Ablations

• Path choice: OT paths are easier to fit than diffusion paths.
• Solver choice: midpoint and RK4 improve the NFE-quality tradeoff over Euler.
• Objective choice: FM outperforms score matching on low-NFE sampling.
• Likelihood estimation: increasing the number of Hutchinson samples improves BPD.

Method Strengths and Weaknesses

Strengths

• Removes ODE simulation from CNF training.
• Unifies diffusion and CNF training through probability-path regression.
• OT paths reduce sampling NFE at similar FID.
• Keeps the standard diffusion U-Net backbone unchanged.

Weaknesses

• Gains depend strongly on choosing a good probability path.
• Sampling still requires ODE solving at test time.
• Main image results reuse diffusion-era U-Net design.
• Likelihood computation still needs divergence estimation and adaptive solvers.

Suggestions from the authors

• Design probability paths directly instead of starting from stochastic diffusions.
• Study path families beyond Gaussian diffusion and OT constructions.
• Extend the framework to broader conditional generation settings.
• Improve efficient CNF likelihood and sampling solvers.

Links

Prior Papers

• @dhariwalDiffusionBeatGANs2021 — supplies the U-Net image backbone and evaluation baseline family.
• @DenoisingDiffusionProbabilisticModels2020 — provides the diffusion objective that FM reinterprets through vector-field regression.
• @songScoreSDE2020 — connects diffusion models to continuous-time dynamics and probability-flow ODEs.
• @dinhNVP2017 — represents the exact-flow lineage that FM extends toward continuous normalizing flows.

Further Papers

• @liuRectifiedFlow2022 — studies straight transport paths for fast generative ODE sampling, closely aligned with FM's path-design view.
• @albergoStochasticInterpolants2023 — generalizes interpolation-based training objectives for transport and diffusion-style generative models.
• @karrasEDM2022 — optimizes diffusion design choices and remains relevant to FM's claim that path design matters.
• @songConsistencyModels2023 — pursues few-step generation, the same efficiency target highlighted by FM's low-NFE results.