Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data
The paper proposes a model that turns live session data into temporary weight changes instead of repeatedly carrying it in the prompt.
A compact hypernetwork generates low-rank modulations of a shared base model’s feed-forward weights from information supplied at run time. The authors say the model keeps a Bayesian belief over the generator’s latent code and updates it online as the session continues. That lets the effective weights change across turns while the stored model size stays fixed. They argue this could save context, amortize compute, persist user-supplied knowledge, and generalize better than in-context learning or retrieval. HN · ArXiv's note
A compact hypernetwork generates low-rank modulations of a shared base model’s feed-forward weights from information supplied at run time. The authors say the model keeps a Bayesian belief over the generator’s latent code and updates it online as the session continues. That lets the effective weights change across turns while the stored model size stays fixed. They argue this could save context, amortize compute, persist user-supplied knowledge, and generalize better than in-context learning or retrieval. HN · ArXiv's note
score 5