← All visualizations
Interactive note 004 Created with Claude Opus 5

A visual exploration of flow matching

Draw one straight line from noise to a data point. Average those lines. Follow the average, and noise becomes structure.

01

The probability path

A line you can draw, a field you can learn

Pairing a noise sample with a data sample gives a path so simple its velocity is a constant. The field we actually learn is what those paths look like on average—and it is the one that transports the whole distribution.

step 0 / 24 0 field evaluations
One straight conditional path
  • conditional
  • marginal
  • sample

Click inside the frame — in Sketch your own it places data in every tab; otherwise it promotes a pair in Path and re-aims the field in Conditional.

Pick a pair

Noise in, data out, straight through

Choose a noise sample and a data sample and simply interpolate between them. The resulting conditional path is Gaussian at every time, and its velocity does not depend on where you are along it.

\[x_0\sim\mathcal N(0,I),\qquad x_1\sim q_{\text{data}}\] \[x_t=\sigma_t\,x_0+t\,x_1,\qquad \sigma_t=1-(1-\sigma_{\min})t\]

The bold pair is one draw of (x0, x1); the faint ones are others. Along a dashed line the orange velocity is the same vector at every time.

data points 96 path width 1.00 sample spread —
02

The training trick

Regress on the line, get the average for free

The marginal field is an intractable integral, so nobody regresses on it. Regressing on the single conditional vector instead works because least squares returns a conditional expectation—and that expectation is the marginal field.

drag the query point x = (−0.35, 0.55) marginal velocity —

Two losses, one gradient

The target you cannot compute

The flow matching loss needs ut(x); the conditional loss needs only the vector between a sampled pair. Expanding both squares shows they differ by a term that does not contain θ, so their gradients agree exactly.

\[\mathcal L_{\text{FM}}(\theta)=\mathbb E_{t,\,x\sim p_t} \left\lVert v_\theta(x,t)-u_t(x)\right\rVert^2\] \[\mathcal L_{\text{CFM}}(\theta)=\mathbb E_{t,\,x_1,\,x\sim p_t(\cdot\mid x_1)} \left\lVert v_\theta(x,t)-u_t(x\mid x_1)\right\rVert^2\] \[\nabla_\theta\mathcal L_{\text{FM}}=\nabla_\theta\mathcal L_{\text{CFM}}\]

What separates the two is the spread of the orange dots around the cross—the variance of the conditional target at this x. It is irreducible noise in the regression, and it is constant in θ:

\[\mathcal L_{\text{CFM}}-\mathcal L_{\text{FM}} =\mathbb E\!\left[\operatorname{Var}\!\left(u_t(x\mid x_1)\mid x_t=x\right)\right]\]

Right panel: every orange dot is one conditional target, sized by its posterior weight. The loss is a quadratic bowl whose minimiser is their weighted mean.

conditional variance — effective data points —
03

The application

Two dimensions, or a hundred and ninety-six

Nothing above cared that the data lived in a plane. Treat a 14 × 14 image as one point in ℝ196, keep the same path and the same field, and the ODE turns static into digits—conditioned on a class, with a step budget you choose.

step 0 / 8 t = 0.00
Condition on

Click any tile to follow that sample in the filmstrip.

Same recipe, bigger vector

Static, resolved

Each tile is a single point moving through pixel space. At t = 0 it is Gaussian noise; the field pulls it toward the data manifold, and the posterior over which image it is heading for sharpens as time runs out. Conditioning simply restricts that posterior to one class.

\[x\in\mathbb R^{196},\qquad x_1\sim q_{\text{data}}(\,\cdot\mid c)\] \[x^{(k+1)}=x^{(k)}+\tfrac1N\,v_\theta\!\left(x^{(k)},\tfrac kN\,\middle|\,c\right)\]

Drop the budget to one or two steps and the tiles come out blurred: a single Euler step lands on the average of the images still in contention. More steps let the field commit before you stop.

Honest caveat: with 50 stored images and the exact field, this memorises—each sample converges to a training image. A learned vθ is a smooth approximation, and that smoothing is where novel samples come from.

dimensions 196 distinct classes — distance to nearest image —
Where this goes

Real image models replace the 50 stored pictures with a network trained by exactly the loss in chapter 02, and the same construction now drives text-to-image and video systems as well as robot policies, where a chunk of future actions plays the role of the image — see Esser et al., Scaling Rectified Flow Transformers (2024) and Black et al., π₀ (2024).