Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers
The paper’s hardware move is to perturb the computation, not the stored weights.
The authors propose IPZO, a zeroth-order fine-tuning architecture for spiking transformers on in-memory computing accelerators. Its perturbation generation unit adds event-triggered perturbation sums alongside IMC weighted sums, avoiding repeated read-modify-write weight updates. A PGU-XOR variant is reported to nearly match software RNG accuracy on Spikingformer/CIFAR-10 and SpikeGPT/WikiText-2, while direct RNG reuse degrades both results. In 16-nm CMOS estimates, PGU-XOR costs more area and per-operation energy than reuse, but converges faster enough to cut total perturbation energy at iso-accuracy. ArXiv · AI/CL/LG's note
The authors propose IPZO, a zeroth-order fine-tuning architecture for spiking transformers on in-memory computing accelerators. Its perturbation generation unit adds event-triggered perturbation sums alongside IMC weighted sums, avoiding repeated read-modify-write weight updates. A PGU-XOR variant is reported to nearly match software RNG accuracy on Spikingformer/CIFAR-10 and SpikeGPT/WikiText-2, while direct RNG reuse degrades both results. In 16-nm CMOS estimates, PGU-XOR costs more area and per-operation energy than reuse, but converges faster enough to cut total perturbation energy at iso-accuracy. ArXiv · AI/CL/LG's note
score 4