Building AI Research Teams for Autonomous Discovery of Linear Attention Architectures
The team discovered linear attention architectures that outperformed GDN2 at 1.3B scale across language modeling, common-sense reasoning, and retrieval tasks.
The researchers gave persistent AI agents different disciplinary backgrounds, private research histories, and shared experimental evidence. An adaptive allocation system selected which agents could propose and run experiments under a limited budget. Removing any one component weakened the discoveries: agents without disciplinary backgrounds plateaued early, those without private histories made less progress, and uniform selection underperformed adaptive allocation.
HF Daily Papers' note
The researchers gave persistent AI agents different disciplinary backgrounds, private research histories, and shared experimental evidence. An adaptive allocation system selected which agents could propose and run experiments under a limited budget. Removing any one component weakened the discoveries: agents without disciplinary backgrounds plateaued early, those without private histories made less progress, and uniform selection underperformed adaptive allocation.
HF Daily Papers' note
score 5