Severe but Harmless: Stress-Testing Shift Severity for Frozen Time-Series Forecasters
Model degradation reached 29% in the regime scored as the least severe distribution shift.
The researchers tested three statistical severity measures across nine regime shifts in three standard traffic datasets. For instance-normalized time-series foundation models, the highest severity scores did not identify the shifts that caused the most degradation. Most comparisons showed no positive relationship between measured severity and degradation. The paper offers an equivariance-based analysis explaining how severity scores can become disconnected from model performance.
ArXiv · AI/CL/LG's note
The researchers tested three statistical severity measures across nine regime shifts in three standard traffic datasets. For instance-normalized time-series foundation models, the highest severity scores did not identify the shifts that caused the most degradation. Most comparisons showed no positive relationship between measured severity and degradation. The paper offers an equivariance-based analysis explaining how severity scores can become disconnected from model performance.
ArXiv · AI/CL/LG's note
score 5