Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data
The paper proposes a model that turns live session data into temporary weight changes instead of repeatedly carrying it in the prompt.
The authors describe an “Infinite-Parameter LLM” built around a compact hypernetwork that generates low-rank modulations of a shared base model from run-time data. A Bayesian belief over the generator’s latent code is updated online, so the effective weights can change as the interaction continues. The claimed payoff is lower repeated prompt cost, more free context, persistence across turns, and better generalization for user-supplied facts or corrections. The paper also lays out an evaluation protocol against in-context learning and retrieval. ArXiv · AI/CL/LG's note
The authors describe an “Infinite-Parameter LLM” built around a compact hypernetwork that generates low-rank modulations of a shared base model from run-time data. A Bayesian belief over the generator’s latent code is updated online, so the effective weights can change as the interaction continues. The claimed payoff is lower repeated prompt cost, more free context, persistence across turns, and better generalization for user-supplied facts or corrections. The paper also lays out an evaluation protocol against in-context learning and retrieval. ArXiv · AI/CL/LG's note
score 6