Hardware Mechanisms to Dynamically Throttle AI Performance
The paper proposes GPU-level controls that can cut an AI workload’s performance without shutting the chip down.
The authors test throttles in the memory subsystem and identify four main levers: L2 size, L2 latency, L2 bandwidth, and shared-memory port access rate. They build the mechanisms from existing hardware techniques, including cache way masking, rate limiting, latency insertion, and bank arbitration. In their evaluation, reducing resources to one-eighth can cut performance by up to 80%, with under roughly 10K flip-flops of added cost. Some combinations degrade performance more than any single knob alone. ArXiv · AI/CL/LG's note
The authors test throttles in the memory subsystem and identify four main levers: L2 size, L2 latency, L2 bandwidth, and shared-memory port access rate. They build the mechanisms from existing hardware techniques, including cache way masking, rate limiting, latency insertion, and bank arbitration. In their evaluation, reducing resources to one-eighth can cut performance by up to 80%, with under roughly 10K flip-flops of added cost. Some combinations degrade performance more than any single knob alone. ArXiv · AI/CL/LG's note
score 4