Five Hypotheses for Why LLMs Fail at Tabular Data.

Feed an LLM a table of training examples and ask it to classify a new row, using nothing but in-context learning (no fine-tuning, just labeled examples in the prompt), and in two dimensions it works shockingly well, agreeing with a Gaussian process 91.6% of the time. Add more columns and it falls apart, becoming the only method out of nine tested that gets worse as you add features. Every classical baseline, from logistic regression to random forests, stays flat or improves. This paper runs down five plausible explanations for the LLM’s weakness on tabular data one at a time, rejects four of them with controlled experiments, and pins the failure specifically on dimensionality.

• Only 1 of 9 methods degrades with more features: the LLM, at a slope of -0.009 accuracy per added dimension, while the best classical baseline improves at +0.012
• In 2D, near-perfect agreement with distance-based methods: 91.6% grid agreement with a Gaussian process (Matérn kernel), 91.0% with 1-nearest-neighbor
• In high dimensions, no classical model can fake it: the best of 252 configured classical baselines matches the LLM’s predictions only 64.8% of the time, even after injecting matched label noise
• The model’s stated reasoning is wrong even when its answer is right: explanations mismatch the data in 14 of 20 two-dimensional tasks
• A single evaluation run cost ~$836 in input tokens alone, on toy-sized datasets, a preview of why this approach doesn’t scale to real tabular data with millions of rows

Ruling out the easy explanations first

Before blaming dimensionality, the authors tested four other suspects, each with a targeted intervention rather than a guess.

Non-linear separability? Rejected. If classes overlap in a way only a curved decision boundary can separate, you’d expect the LLM to struggle more as separability drops. It barely moved (slope +0.089) while logistic regression responded four times as strongly (+0.368), so this isn’t the bottleneck; if anything the LLM is oddly insensitive to it.

CSV formatting hides the right column? Rejected. A “needle in the haystack” test buried the relevant column at increasing depth in a wide CSV; performance stayed flat out to 60 columns deep (r=-0.17). The model reliably finds and reads the column it needs regardless of position.

Numeric tokenization bloats the prompt? Rejected. Long decimal mantissas get chopped into more tokens, which could in principle dilute attention. Truncating precision had essentially no effect (r=-0.02).

Less compute per test point at scale? Rejected. Normalized score stayed put across full, half, and quarter-sized test batches (0.422, 0.412, 0.407), so the LLM isn’t just running out of budget per query.

The dimensionality effect is unique to the LLM

The core experiment sweeps random linear projections across 31 datasets, adding features one at a time, and tracks accuracy for nine methods side by side: the LLM, plus classical baselines including logistic regression, random forests, SVMs, and k-NN. Every classical method’s trend line is flat or rising as dimensions increase. The LLM is the only one that reliably falls.

This is a familiar shape in machine learning: the curse of dimensionality. As you add features, the volume of the space grows exponentially, so a fixed number of training examples covers an ever-shrinking fraction of it, and points that were “close” in a low-dimensional sense become nearly equidistant from everything. Distance-based methods like k-nearest-neighbors are famously vulnerable to this; classical algorithms like random forests and SVMs were specifically engineered, through feature selection, regularization, margins, to resist it. The LLM behaves like an un-regularized distance-based method here. It has no built-in defense against the curse, because nothing in its training ever taught it one.

What the model is actually doing

In two dimensions, the LLM’s predictions track a Gaussian process with a Matérn kernel almost exactly (91.6% grid agreement) and closely resemble 1-nearest-neighbor (91.0%). That’s a strong signal: at low dimension, the model appears to be doing something like local, distance-based reasoning over the examples in its context, “what’s near this point tends to share its label,” rather than fitting a global decision rule.

That story breaks down completely once dimensionality rises. The authors checked whether the collapse could simply be explained as “the LLM’s underlying strategy still works, it’s just noisier now,” by taking the best of 252 configured classical baselines and injecting label noise scaled to match the LLM’s degraded accuracy. Even with noise added, the noisy classical model only agreed with the LLM’s actual predictions 64.8% of the time, and better noise-matching improved that by at most 0.64 percentage points. Whatever the LLM is doing wrong in high dimensions isn’t “the same algorithm, executed worse.” It’s a different, currently unidentified failure mode.

None of this would mean much if the LLM were simply reciting memorized answers, since datasets like iris and breast cancer are famous enough to plausibly appear in training data. So before trusting any of it, the authors ran a contamination check: hold out one entire class during the “training” portion of the prompt, then see if the model still predicts it on test rows, something a genuine in-context learner should score zero on. It didn’t. The model recovered the true labels almost perfectly on the held-out class, a clean tell for memorization, which is why the contaminated datasets were excluded from the main analysis above.

Right answer, wrong reason

A separate experiment asked the model to explain its reasoning on the 2D tasks where it performs well. In 14 of 20 tasks, the stated explanation didn’t match the actual data: the model would describe a decision rule that, if you actually applied it, wouldn’t produce the label it just gave. The prediction and the explanation appear to come from different processes, direct evidence that the model’s stated reasoning isn’t “faithful” to whatever computation actually produced the answer. That matters beyond this paper: you can’t safely use a model’s explanation to debug or audit its behavior if the explanation isn’t causally connected to how it got there.

Caveats

This is a behavioral study, not a mechanistic one: the authors watch inputs and outputs, they don’t have access to model internals, so “why” the collapse happens at the representation level is still an open question. The primary results rest on one model (Claude Opus 4.6), with only limited cross-checking on Qwen for the 2D tasks, so it’s unclear how universal this specific failure curve is across model families and future versions. And everything here runs on toy-scale datasets, both because of context window limits and because a single run already cost roughly $836 in input tokens alone; real tabular datasets with millions of rows and hundreds of columns are well outside what this method could touch today regardless of the accuracy question.

None of that undercuts the core result: whatever makes classical tabular ML resistant to added features, LLMs doing in-context learning don’t have it, and their explanations can’t be trusted to tell you why.