iFAN: Inference-Aware Learning for Plain Mask Transformers
The paper’s claim is that mask transformers can be trained to match the way they actually choose masks at inference time.
iFAN targets two training-inference mismatches: confident query masks can be wrong, and useful intermediate-layer predictions can be lost before final decoding. Its APMR objective ranks query competition by adjusted mask quality, while CLSD distills stronger intermediate predictions into the final layer. The authors say these objectives are used only during training, leaving final-layer inference essentially unchanged. Reported gains average 1.20 PQ, 1.30 AP, and 0.63 mIoU across COCO, ADE20K, and Cityscapes, with negligible added parameters, FLOPs, or latency. HF Daily Papers' note
iFAN targets two training-inference mismatches: confident query masks can be wrong, and useful intermediate-layer predictions can be lost before final decoding. Its APMR objective ranks query competition by adjusted mask quality, while CLSD distills stronger intermediate predictions into the final layer. The authors say these objectives are used only during training, leaving final-layer inference essentially unchanged. Reported gains average 1.20 PQ, 1.30 AP, and 0.63 mIoU across COCO, ADE20K, and Cityscapes, with negligible added parameters, FLOPs, or latency. HF Daily Papers' note
score 4