Memory Attention
The paper replaces attention’s value projection with token-indexed memory plus contextual keys.
Memory Attention stores layer-specific token representations in memory tables, while keys keep the values context-aware. At inference, its normalization can be folded into those tables, making value construction a lookup-and-add step. The author says this can support CPU offloading with prefetching, cutting GPU parameter storage. In matched-token experiments with extra memory parameters, the method improved language modeling and average downstream results. ArXiv · AI/CL/LG's note
Memory Attention stores layer-specific token representations in memory tables, while keys keep the values context-aware. At inference, its normalization can be folded into those tables, making value construction a lookup-and-add step. The author says this can support CPU offloading with prefetching, cutting GPU parameter storage. In matched-token experiments with extra memory parameters, the method improved language modeling and average downstream results. ArXiv · AI/CL/LG's note
score 5