Memorilla: Latent Semantic Memory for LLMs
With 16 memory tokens, Memorilla outperformed comparison systems on five of eight benchmarks.
The system encodes documents once into reusable embeddings, then gathers query-specific information into the input of a frozen LLM. The authors compared it with baselines including Mem0 and MemoRAG across personalization and question-answering tasks. On FactKG, it reached 84.33% accuracy versus 80.43% for full-context inference, using 108 times fewer tokens to represent source documents. In TextWorld, it reduced revisits to explored rooms and raised average agent success from 18.6% with full-history prompting to 24.2%.
HN · ArXiv's note
The system encodes documents once into reusable embeddings, then gathers query-specific information into the input of a frozen LLM. The authors compared it with baselines including Mem0 and MemoRAG across personalization and question-answering tasks. On FactKG, it reached 84.33% accuracy versus 80.43% for full-context inference, using 108 times fewer tokens to represent source documents. In TextWorld, it reduced revisits to explored rooms and raised average agent success from 18.6% with full-history prompting to 24.2%.
HN · ArXiv's note
score 5