Non-Commutative State Tracking with Input-Dependent Low-Rank Updates in Mamba-3
The paper adds an input-dependent rank-one reflection to Mamba-3 so it can track states where operation order changes the result.
Fujii and Yamakita keep Mamba-3’s discretization, RoPE, and readout, but replace purely diagonal transitions with a low-rank term that couples state coordinates. They adapt chunkwise computation so the recurrence can still be parallelized within chunks. Tests include group word problems and a continuous-observation shell game trained by behavioral cloning. In the shell game, the proposed model held higher tracking success on longer swap sequences with timing jitter than standard Mamba-3. ArXiv · AI/CL/LG's note
Fujii and Yamakita keep Mamba-3’s discretization, RoPE, and readout, but replace purely diagonal transitions with a low-rank term that couples state coordinates. They adapt chunkwise computation so the recurrence can still be parallelized within chunks. Tests include group word problems and a continuous-observation shell game trained by behavioral cloning. In the shell game, the proposed model held higher tracking success on longer swap sequences with timing jitter than standard Mamba-3. ArXiv · AI/CL/LG's note
score 5