A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization
The paper splits LLM optimization into structure generation and separate numeric tuning.
Gallego frames the LLM as the outer solver: it proposes sketches with numeric gaps instead of trying to tune every parameter itself. An inner optimizer then fills those gaps using methods such as CMA-ES, gradient routines, or MCMC samplers. The framework is tested on meta-optimization, code-based policies, and approximate Bayesian inference, where it beats both vanilla LLM search and pure numerical baselines. HF Daily Papers' note
Gallego frames the LLM as the outer solver: it proposes sketches with numeric gaps instead of trying to tune every parameter itself. An inner optimizer then fills those gaps using methods such as CMA-ES, gradient routines, or MCMC samplers. The framework is tested on meta-optimization, code-based policies, and approximate Bayesian inference, where it beats both vanilla LLM search and pure numerical baselines. HF Daily Papers' note
score 4