EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
EngramEdit updates facts by editing conditional memory while leaving the Transformer backbone fixed.
The method computes target memory representations for an updated fact across multiple expressions, then adjusts shared n-gram embeddings toward those targets. It penalizes changes to frequently reused embeddings to limit spillover into unrelated knowledge. The paper reports near-perfect editing success, stronger use of revised facts in unseen phrasing and multi-hop reasoning, and roughly three times the best baseline’s chain-of-thought accuracy. General capabilities and unrelated knowledge are described as largely preserved as edits accumulate. HF Daily Papers' note
The method computes target memory representations for an updated fact across multiple expressions, then adjusts shared n-gram embeddings toward those targets. It penalizes changes to frequently reused embeddings to limit spillover into unrelated knowledge. The paper reports near-perfect editing success, stronger use of revised facts in unseen phrasing and multi-hop reasoning, and roughly three times the best baseline’s chain-of-thought accuracy. General capabilities and unrelated knowledge are described as largely preserved as edits accumulate. HF Daily Papers' note
score 4