LLMs as Feature Engineers for Text-and-Tabular Prediction
The paper’s loop turns model mistakes into instructions for LLM-generated categorical features.
Barlier and Skrlj describe a system where one LLM proposes schema-bound feature definitions from text, another extracts them, and a tabular model tests whether they help prediction. The feedback comes from explicit errors such as AUC ranking inversions, rewritten as natural-language guidance for the next feature search. Across three public datasets, the authors report up to a 3x speedup over unguided search. The generated features also rank highly under SHAP and are presented as an instance-level audit trail. ArXiv · AI/CL/LG's note
Barlier and Skrlj describe a system where one LLM proposes schema-bound feature definitions from text, another extracts them, and a tabular model tests whether they help prediction. The feedback comes from explicit errors such as AUC ranking inversions, rewritten as natural-language guidance for the next feature search. Across three public datasets, the authors report up to a 3x speedup over unguided search. The generated features also rank highly under SHAP and are presented as an instance-level audit trail. ArXiv · AI/CL/LG's note
score 5