Complementary Roles of Activation and Parametric Memory in Few-Shot Learning
The paper finds that a few-shot model needed both KV-cache history and parameter updates to solve its Conditional Arithmetic task.
Activation memory performed better for factual recall, while parametric memory did not reliably beat it on task learning. In neuron-level tests, the authors found different neuron sets firing depending on whether the same information was accessed through activations or parameters. Combining the two memory paths recruited both sets, which the paper says was crucial for the composite task. ArXiv · AI/CL/LG's note
Activation memory performed better for factual recall, while parametric memory did not reliably beat it on task learning. In neuron-level tests, the authors found different neuron sets firing depending on whether the same information was accessed through activations or parameters. Combining the two memory paths recruited both sets, which the paper says was crucial for the composite task. ArXiv · AI/CL/LG's note
score 4