Cortex: Consolidating Context into Parametric Memory
After 50 sequential factual edits, Cortex-Structural recovered 88.9% of the context-induced gain while retaining 99.8% of original non-target performance on a 14B model.
Cortex aims to preserve improvements from supplied context after that context is removed. It generates queries anticipating future uses of the context, then distills context-conditioned outputs into mergeable model-weight updates. The authors evaluate five task families with single, batch and sequential updates across 4B–120B text-only and multimodal models. They report that Cortex offers one of the best measured tradeoffs between consolidation and retention in most evaluated settings.
HF Daily Papers' note
Cortex aims to preserve improvements from supplied context after that context is removed. It generates queries anticipating future uses of the context, then distills context-conditioned outputs into mergeable model-weight updates. The authors evaluate five task families with single, batch and sequential updates across 4B–120B text-only and multimodal models. They report that Cortex offers one of the best measured tradeoffs between consolidation and retention in most evaluated settings.
HF Daily Papers' note
score 5