2608.07294-FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching".

Jane: The paper was written by Suman Cha, Seongchan Lee, Dohyun Ko and Hyunjoong Kim from Yonsei University and KAIST.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper summary: Tom: Today's paper comes from Yonsei University and KAIST, and it's called FUSE — Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching. The authors are Suman Cha, Seongchan Lee, Dohyun Ko, and Hyunjoong Kim. In one sentence, they've built a new neural architecture for generating synthetic tabular data, and it posts consistently strong results across eight benchmark datasets.

Jane: What grabs me is the split in the architecture before we even look at those numbers. One module specializes the computation per feature type, and a separate mechanism handles exchange across columns.

Tom: That split is the whole thesis. And the results back it up — best average rank on marginal fidelity at 1 point 25, and best rank on the multivariate classifier test at 1 point 62.

Lu: Synthetic tabular data matters because hospitals and banks use it to share records without exposing real people, and it can help when your training set is thin. Data sharing, scarcity, and robust downstream analysis are the motivations the paper lists up front.

Meng: The hard part is that tables mix very different kinds of variables. A column for income behaves nothing like a column for employment status, and even two numerical columns can look completely different — one heavily skewed, another multimodal.

Jane: The paper builds on variational flow matching, where each column gets its own endpoint distribution. Numerical columns get Gaussian factors, categorical columns get categorical ones. But the network behind it all is a shared backbone, and feature identity is just an embedding.

Tom: So the loss treats columns individually, but the computation doesn't.

Lalam: That's the gap FUSE closes. Type-specific adaptive mixture modules let each feature combine shared subnetworks in its own way, and joint self-attention lets numerical and categorical columns exchange information across the entire row. Specialization and exchange become explicit, separate mechanisms.

Tom: There's a theoretical side as well. They show what you pay when a predictor can't condition on the full context, and they bound the Wasserstein distance between generated and real data in terms of endpoint prediction risk.

Lalam: An architectural idea, supporting theory, and consistent experimental gains — that combination is why this paper deserves a careful read.

Jane: The introduction on page one explains why mixed-type generation is still hard, so let's start there.

Page 1: Tom: We've heard the one-sentence version of the paper. The introduction on page one lays out exactly why mixed-type generation remains hard.

Jane: Page one opens with the motivation — tabular data dominates machine learning applications in healthcare, finance, and public policy, and synthetic copies make data sharing and augmentation possible without exposing real records. The paper also brings up mitigating data scarcity and enabling robust downstream analysis.

Lu: Then comes a point that shapes the whole paper. The difficulty isn't just numerical versus categorical. Columns within the same type have different structures — a numerical column can be skewed, heavy-tailed, or multimodal, and categorical columns differ in cardinality and in how concentrated their frequencies are.

Tom: So even two columns of the same type shouldn't necessarily get the same treatment.

Meng: The paper walks through the flow matching lineage. Flow matching itself learns a time-dependent velocity field, and integrating that field transports a simple source distribution to the data distribution.

Jane: Variational flow matching expresses that velocity through inference over endpoints, and the mean-field version factorizes that inference across columns. Then exponential-family VFM assigns appropriate distributions to each variable type.

Lu: Each step makes the objective more column-aware, which is exactly why the architectural gap becomes visible. The factorization aligns the objective with the data types, but nothing in the network does.

Tom: Which brings us to TabbyFlow, the direct predecessor. It implements EF-VFM with a shared backbone, processing column embeddings through a common network that doesn't distinguish feature types in its computation.

Jane: The authors' argument is that column embeddings encode identity, but they don't provide any explicit mechanism for feature-dependent transformations. That's the gap.

Lalam: So the motivation is architectural. The loss already respects column types, but the network doesn't. And the fix has to preserve parameter sharing while adding specialization — which is exactly the balance FUSE strikes.

Tom: The introduction closes with three contributions — the FUSE architecture, theory around restricted conditioning and Wasserstein generation error, and the experimental evaluation.

Jane: Before we can judge those contributions, page two builds the formal framework underneath.

Page 2: Tom: Page two supplies the formal machinery. The endpoint X1 stacks the numerical columns together with one-hot vectors for the categorical columns, so the overall space has dimension equal to the numerical count plus all the category counts.

Jane: The source distribution is standard Gaussian on that space, and the conditional path is linear — Xt is one minus t times the noise plus t times the data point. The conditional velocity is simply the endpoint minus the state, divided by one minus t.

Lu: The clean consequence follows immediately. The marginal velocity only depends on the endpoint posterior through its mean, so everything else about the posterior is irrelevant to the flow.

Meng: That's what makes variational flow matching tractable. Instead of modeling the full joint posterior over endpoints, you approximate it, and the mean-field version factorizes that approximation across columns.

Tom: And exponential-family VFM picks the right family per column — Gaussian factors for numerical endpoints, categorical factors for categorical ones, with a predetermined variance schedule.

Jane: So what does the training objective look like? It's a mean-squared error for the numerical columns plus a cross-entropy term for each categorical column. The categorical term is the log-probability of the true category under the predicted distribution.

Tom: Here's the subtle part. The mean-field assumption factorizes the variational distribution across columns, but it places no restriction on the conditioning argument. Each predictor can still see every coordinate of the intermediate state.

Lalam: That distinction matters. The factorization shapes the output distribution, but the network's capacity to combine information is left completely open.

Meng: And the paper says it plainly — the variational objective does not specify how computation should be shared among these column-wise parameter functions. That's the slot FUSE fills.

Jane: Yes. Page three shows the architecture that fills that slot.

Page 3: Tom: Page three introduces the architecture, and the first thing to notice is that every feature becomes a token. A feature-specific linear projection, plus a time embedding, tells the network where it is along the flow.

Jane: The adaptive mixture module is the centerpiece. It computes alignment scores between feature tokens and a set of latent components, then normalizes those scores along two axes.

Lu: Two normalizations, two different jobs. Aggregation weights, normalized across features, decide how tokens get pooled into components. Recombination weights, normalized across components, decide how each feature pulls information back out.

Meng: So a feature like age sends a weighted mixture of its own representation into a handful of shared subnetworks, and the output comes back recombined according to a different set of weights. Different columns end up using the subnetworks in different proportions.

Tom: And the subnetworks are shared across all features of the same type, which keeps the parameter count under control. You get specialization without a separate network for each column.

Jane: The numerical and categorical types never share subnetworks with each other, which is where the type-specificity comes from.

Lalam: Then joint attention takes the processed numerical and categorical tokens, concatenates them, and applies standard multi-head self-attention. That's the exchange mechanism — every column can condition on every other column.

Tom: The two roles are now explicit. Mixtures for specialization, attention for exchange.

Jane: Endpoint heads read out the numerical means and categorical probabilities from the final tokens, and those predictions define the velocity field.

Meng: One detail I appreciate is that the normalization in each branch is time-conditioned, so the balance between components can vary along the flow. The same architecture can behave differently early and late in generation.

Tom: The architecture is in place. Page four asks what goes wrong if you remove either piece.

Page 4: Tom: Page four is theory. Proposition one quantifies the cost of restricted conditioning — if a predictor can't see some features, the excess risk decomposes into a numerical term plus a sum of KL divergences.

Jane: The numerical term is the expected squared gap between the full and restricted posterior means, scaled by the noise variance. Each categorical term measures how much posterior information about that category is lost.

Lu: Then come two bottleneck examples that separate two failure modes.

Tom: The conditioning bottleneck shows that an informative numerical proxy isn't sufficient. If an endpoint's mean depends on a latent sign, and the predictor only sees a noisy version of that sign, the excess risk is strictly positive — they compute it as a constant times the expected variance of the sign given the proxy.

Meng: The representation bottleneck is about shared computation. Two endpoint means that live in orthogonal directions can't be captured by a single shared scalar representation; the minimum error is the smaller of the two squared coefficients.

Jane: But a shared dictionary with two components represents both means exactly, as long as features use different recombination weights. The paper gives explicit weights — three quarters and one quarter, swapped between the two features.

Lalam: So the two components of FUSE map onto two distinct failure modes. Attention addresses the conditioning bottleneck, and mixtures address the representation bottleneck.

Tom: Theorem one then connects endpoint risk to generation quality. Under Lipschitz and integrability assumptions, the Wasserstein error is bounded by the square root of the excess risk, scaled by a constant that depends on the time horizon, plus a truncation term.

Jane: The trade-off is explicit. Larger T shrinks the truncation term because the flow gets closer to the final time, but it inflates the coefficient in front of the risk term.

Lu: The appendix verifies that a fixed FUSE network satisfies the regularity conditions — the vector field is bounded, Lipschitz, and integrable at the origin. So the bound applies to their actual model.

Tom: With the theory in hand, the paper moves to experiments. Page five sets up the evaluation.

Page 5: Tom: Page five sets up the experimental campaign. Eight datasets — Adult, Default, Beijing, Shoppers, Magic, News, Diabetes, and Fault — spanning binary classification, multiclass classification, and regression.

Jane: Sample sizes range from 691 to 37,581. Seven baselines across four families: CTGAN for GANs, TVAE for VAEs, CoDi, TabDDPM, TabSyn, and TabDiff for diffusion, and TabbyFlow for flow matching.

Lu: Four metrics cover different quality dimensions. Shape measures marginal fidelity with Kolmogorov-Smirnov statistics for numerical features and total variation distances for categoricals.

Meng: Trend measures pairwise dependencies by comparing correlation structures. C2ST trains an XGBoost classifier to tell real from synthetic, and MLE trains a model on synthetic data and scores it on held-out real data.

Tom: The headline is Shape, where FUSE ranks first on six datasets and second on the other two — an average rank of 1 point 25, against 2 point 50 for TabbyFlow.

Jane: That gap is striking, and it's consistent. The improvement appears on every dataset, not just one or two favorable ones.

Lalam: Shape is also the metric most directly tied to their design. If type-specific processing works, it should show up first in the marginals.

Tom: The main tables average over twenty random seeds, and the implementation uses a hidden dimension of 256 with four layers and four attention heads, trained with AdamW. So these aren't cherry-picked runs.

Jane: Page six asks whether the advantage survives when the metrics look at dependencies and downstream tasks.

Page 6: Tom: Page six reports Trend, C2ST, and MLE. On Trend, FUSE has the highest score on five datasets and the best cross-dataset mean, 0 point 982 against 0 point 978 for TabbyFlow.

Jane: The largest Trend gains over TabbyFlow are on Diabetes and Fault — from 0 point 936 up to 0 point 966, and from 0 point 978 up to 0 point 985.

Lu: But TabbyFlow still holds a slightly better average rank on Trend, 1 point 75 to 2 point 00, because it ranks higher on Beijing, Magic, and News. So the pairwise dependence story is competitive rather than dominant.

Tom: C2ST is where FUSE stands out. Average rank 1 point 62, first on Default, Beijing, and News, second on the remaining five. It's the only method that places in the top two on all eight datasets.

Meng: That consistency matters because C2ST uses all features jointly. If FUSE only nailed univariate marginals, a classifier looking at the full row would still find discrepancies.

Jane: On MLE, FUSE has the best average rank at 2 point 62, and the standout result is News — RMSE drops from 0 point 872 for TabbyFlow to 0 point 841. That's roughly a 3 point 6 percent relative improvement.

Tom: The visualizations back up the tables. Correlation error heatmaps show FUSE at near-zero errors on News, while other methods show widespread discrepancies.

Lu: There are wins on Shoppers and Fault too, plus a second-best on Default — the utility gains aren't concentrated in one dataset.

Tom: So the aggregate picture is strong. The next page shows which components drive it.

Page 7: Tom: Page seven contains the component analysis. Six configurations vary whether the mixture modules are active, whether attention is joint or restricted, and whether mixtures apply to numerical, categorical, or both types.

Jane: The joint attention effect is dramatic under dense processing. Enabling it lifts Trend from 0 point 960 to 0 point 976, and AUC from 0 point 630 to 0 point 888 — that's a massive jump in downstream utility.

Lu: With both mixture modules active, the comparison is similar. Trend goes from 0 point 960 to 0 point 977, AUC from 0 point 644 to 0 point 882, and RMSE from 0 point 818 down to 0 point 725.

Meng: The adaptive mixture modules contribute elsewhere. Under joint attention, they push Shape from 0 point 984 to 0 point 985, C2ST from 0 point 985 to 0 point 991, and β-Recall from 0 point 482 to 0 point 545.

Tom: So the picture is clear. Attention is the workhorse for dependencies and downstream utility, while the mixtures improve multivariate fidelity and coverage.

Jane: The paper also notes trade-offs. The numerical-only mixture achieves the highest β-Recall, and the categorical-only version matches FUSE on RMSE, but the full configuration has the strongest overall profile.

Lalam: That division of labor was predicted by page four's theory. Attention prevents the conditioning bottleneck, mixtures prevent the representation bottleneck.

Tom: The conclusion draws it together — FUSE makes the parameter-sharing configuration explicit, the theory characterizes the roles of each component, and the experiments confirm they're complementary.

Jane: The supplementary material extends this with a controlled experiment where they dial the cross-type dependence from zero to nearly one. The advantage of joint attention grows right along with it.

Lu: For the field, the larger message is that tabular generation benefits from an architecture that matches the data structure.

Tom: Let's close with what this means beyond the benchmark tables.

Conclusion: Tom: So where does this leave us? FUSE separates feature specialization from cross-column exchange in variational flow matching, and it does so with a clean modular design that's easy to describe.

Jane: The theory gives a vocabulary for why it works. Restricted conditioning costs measurable risk, and endpoint prediction quality directly bounds generation quality. That's a useful bridge between the loss you optimize and the distribution you actually generate.

Lu: The empirical record is strong — best average rank on Shape and C2ST, competitive on Trend and MLE. And the consistency across eight datasets is the most convincing part of it.

Meng: The ablations convince me most. Each component earns its place, and the theory anticipated which metric each component would affect — attention for dependencies, mixtures for fidelity and coverage.

Lalam: For the wider field, this is another sign that tabular generation is moving from generic backbones toward purpose-built designs. When you know the data structure, you should exploit it, and here the structure is mixed types.

Tom: The practical implications reach beyond benchmarks. Better synthetic tables mean safer data sharing in healthcare and finance, and more reliable augmentation when real data is scarce.

Jane: There's room to push further. The theory applies before categorical decoding, and the controlled dependence experiment in the appendix points at a deeper analysis of when attention pays off.

Lu: The paper's own framing is apt — a favorable balance across marginal, pairwise, multivariate, and task-oriented evaluations, without relying on dominance in any single metric.

Meng: Variational flow matching is still young, and this paper shows the objective and the architecture can be developed almost independently. That separation is a useful blueprint.

Tom: For anyone working with synthetic data, this paper is worth a careful read. We'll be watching for follow-ups.

Jane: That closes our discussion. On to the next one.