Gradient-Update Mismatch: Rethinking Conflict-Free Training of Physics-Informed Neural Networks
The paper says “conflict-free” PINN training can become conflicted again once the optimizer turns the gradient direction into an actual update.
The authors call that gap Gradient-Update Mismatch, or GUM. They argue it can arise from momentum, adaptive scaling, preconditioning, historical state, or decoupled weight decay. Their proposed fix, Gradient-Update Alignment, projects the optimizer’s proposed update back into the conflict-free cone and adjusts optimizer state when needed. In their experiments, mismatch conflict rates reached 86.3%, while GUA kept applied updates conflict-free and reduced relative `L2` error by up to 98.2% in individual settings. ArXiv · AI/CL/LG's note
The authors call that gap Gradient-Update Mismatch, or GUM. They argue it can arise from momentum, adaptive scaling, preconditioning, historical state, or decoupled weight decay. Their proposed fix, Gradient-Update Alignment, projects the optimizer’s proposed update back into the conflict-free cone and adjusts optimizer state when needed. In their experiments, mismatch conflict rates reached 86.3%, while GUA kept applied updates conflict-free and reduced relative `L2` error by up to 98.2% in individual settings. ArXiv · AI/CL/LG's note
score 4