GoKawiilBenchmark test: TabPFN and TabICL beat tuned XGBoost on all 14 tabular datasets
An independent test compared pretrained tabular foundation models TabPFN and TabICL, which make predictions without training on new data, against a hyperparameter-tuned XGBoost model. Across 14 datasets from the Grinsztajn benchmark, using the same data splits and timing for all methods, the non-training models outperformed tuned XGBoost on every single dataset, including at scales up to 32,000 rows.