Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers
The proposed swap made a small Transformer faster to generate with, but worse on every reported quality metric.
The paper replaces ordinary dense multiplication in the feed-forward layer with an associative-algebra product that keeps the parameter count roughly fixed. In a paired 110M-parameter decoder-only LM test, the algebraic version showed a 6.2–7.8% end-to-end generation throughput gain across four prompt domains. It also scored lower on all three downstream metrics the authors report. The authors frame this as a small-scale feasibility check, not a win on model quality. HF Daily Papers' note
The paper replaces ordinary dense multiplication in the feed-forward layer with an associative-algebra product that keeps the parameter count roughly fixed. In a paired 110M-parameter decoder-only LM test, the algebraic version showed a 6.2–7.8% end-to-end generation throughput gain across four prompt domains. It also scored lower on all three downstream metrics the authors report. The authors frame this as a small-scale feasibility check, not a win on model quality. HF Daily Papers' note
score 5