How do flow matching models memorize and generalize in sample data subspaces

Abstract Real-world data often lie near a low-dimensional structure embedded in a high-dimensional space. In practice, we observe only finitely many samples, which define a sample data subspace that supports tasks such as dimensionality reduction and generation. We study whether generative models can synthesize samples that remain within this subspace. Using Flow Matching models that transport a simple prior to a complex target via a learned velocity field, and treating the data distribution as discrete, we derive analytical expressions for the optimal velocity field under a Gaussian prior. The result shows that generated samples memorize data points and exactly represent the sample data subspace. To generalize to suboptimal scenarios, we introduce the Orthogonal Subspace Decomposition Network, which splits the velocity into subspace and off-subspace components. Our analysis shows the off-subspace component decays while the subspace component generalizes within the sample data subspace, preserving both proximity and diversity.