Fractional State Space Transition for Long Sequence Modeling
FRAC swaps the usual exponential forgetting in state space models for a power-law memory mechanism aimed at long context.
The paper says modern SSMs often lose distant information because ODE-style dynamics decay too quickly. FRAC uses fractional dynamics, then approximates the heavy-tailed memory with a finite set of log-spaced exponential modes so it can still run as an efficient recurrent module. The authors report gains over current SSM baselines on long-context tests, including at 1.3B-parameter language-model scale, while remaining competitive on short context. HF Daily Papers' note
The paper says modern SSMs often lose distant information because ODE-style dynamics decay too quickly. FRAC uses fractional dynamics, then approximates the heavy-tailed memory with a finite set of log-spaced exponential modes so it can still run as an efficient recurrent module. The authors report gains over current SSM baselines on long-context tests, including at 1.3B-parameter language-model scale, while remaining competitive on short context. HF Daily Papers' note
score 5